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THE  COMPLEXITY  OF  FINITE  FUNCTIONS 


ABSTRACT 


Lower  bounds  on  the  length  of  formulas  for  finite  functions  are 
obtained  from  a  generalization  of  a  theorem  of  Specker.  Let  f: 

{ 0,1,. . . ,d-l]n  { 0,1 , . . .  ,d-l]  be  a  function  which  can  be  represented 
by  a  formula  of  length  ^  c*n.  For  any  m,  if  n  is  sufficiently  large, 
there  is  a  restriction  f':  (0,1,.. . ,d-l]m  -♦  ( 0, 1 , . . . ,d-l]  of  f  which 
is  representable  by  a  special  class  of  formulas  called  homogeneous 
e-complexes.  By  showing  that  certain  functions  do  not  have  restric¬ 
tions  representable  by  homogeneous  e-complexes,  we  are  able  to  conclude 
that  the  length  of  formulas  representing  the  mod  p  sum,  p  >  d,  or  the 
connectedness  of  a  pattern  on  a  discrete  retina  cannot  be  bounded  by 
a  linear  function  of  the  number  of  variables  in  the  formula. 

Also  considered  are  perceptrons  over  finite  fields  (cyclic  per- 
ceptrons).  It  is  shown  that  cyclic  perceptrons  of  bounded  order 
cannot  represent  the  geometric  predicate  connectivity.  An  interesting 
aspect  of  this  is  that  one  proof  of  the  corresponding  result  for 
bounded  order  perceptrons  over  the  rationals  rests  on  the  inability 
of  the  latter  to  represent  the  parity  function.  However,  the  parity 
function  requires  order  1  if  the  field  has  chracteris tic  2;  thus, 
this  proof  breaks  down  in  the  case  of  cyclic  perceptrons.  Another 
geometric  predicate  that  cannot  be  represented  by  bounded  order 
cyclic  perceptrons  is  Euler  number  equals  k  (for  an  arbitrary  k). 
However,  this  predicate  can  be  represented  by  bounded  order  percep¬ 
trons  over  the  rationals.  It  must  be  noted,  however,  that  our  proofs 
are  different  and  much  simpler  than  the  corresponding  proofs  derived 
by  Minsky  and  Paper t  for  perceptrons  over  the  rationals. 

Finally,  we  investigate  k-pattern  spectra  of  a  discrete  retina. 
This  is  the  2^  -tuple,  each  component  of  which  corresponds  to  the 
number  of  times  a  particular  kxk  pattern  occurs  on  the  retina.  It 
is  shown  that  the  only  topological  predicates  that  can  be  determined 
from  k-pattern  spectra  of  discrete  figures  are  functions  of  the  Euler 
number  of  the  figure. 


This  report  reproduces  a  thesis  of  the  same  title  submitted  to 
the  Department  of  Electrical  Engineering,  Massachusetts  Institute 
of  Technology,  in  partial  fulfillment  of  the  requirements  for 
the  degree  of  Doctor  of  Philosophy,  February  1972. 
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CHAPTER  ONE 

INTRODUCTION  AND  SURVEY 

1.1  Finite  Functions 

Let  n  be  nonzero  and  finite;  then  a  partial  function]Wn  -+3KT  ,  defined 
on  only  finitely  many  n-tuples,  is  called  a  finite  function.  We  will  restrict 
our  attention  to  a  subclass  of  finite  functions.  D  =  (0,1,. . . ,d-l]  is  an 
initial  interval  of  JKT.  Then  we  will  consider  the  set  of  all  (total) 
functions  Dn  -»  D  for  all  possible  D  and  (finite  and  nonzero)  n. 

Let  f:  Dn  D.  Then  f  is  identified  with  a  (functional)  table  with  dn 
rows  (corresponding  to  all  possible  n-tuples  over  D)  and  n+1  columns  (corres¬ 
ponding  to  the  n  arguments  and  the  value  of  f).  Obviously  the  number  of  func- 
,n 

tions  Dn  -*  D  is  dd  . 

Consider  any  function  f:  Dn  -♦  D  for  arbitrc  y  D,  n.  We  will  say  that  f 
th 

depends  on  the  i  argument  if  and  only  if  there  exist  two  n-tuples 

il  =  (a,  > • • • ,a  , . . . ,a  )  and  b  =  (b.  , . . .  ,b  , . . .  ,b  )  such  that  a  =b  for  j  Mi 
1  1  n  lin  j  j 

ai  ^  bi’  and  ^  f(b).  Suppose  that  f  does  not  depend  on  its  jtb  argument; 

til 

then  we  will  say  that  the  j  argument  is  a  fictitious  argument. 

1.2  Formulas 

Let  there  be  given  the  countable  sets  S  =  (XpX2,...]  of  variable  symbols 
and  0  of  operator  symbols.  Each  element  of  Q  is  a  name  for  a  function  in 
and  conversely  each  function  in  7  has  a  name  in  fi.  Let  9  f  Q  represent  the 
function  f:  D°  -»  D.  Then  we  will  write  arg(<P)  =  n  and  dom(cp)  =  D. 
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1.2.1  Definition 

A  D -.formula  is  a  finite  expression  F  =  cp^, . . .  ,Gr)  such  that  cp  6  fi  , 
arg(cp)  =  n,  dom(tp)  =  D,  and  either  Gi  €  H  or  Gt  is  a  D-formula  for  1  5  i  £  n, 

A  formula  is  simply  a  D-formula  for  some  D. 

Let  F  be  an  arbitrary  D-formula  and  let  xn  be  the  highest  numbered 
variable  symbol  appearing  in  F.  Then  F  represents  a  function  f:  Dn  ■*  D. 

This  correspondence  is  well-known  and  we  will  not  describe  it  in  detail. 
Without  danger  of  imprecision,  F  will  also  be  considered  as  a  representation 
for  all  functions  obtained  from  f  by  adding  fictitious  arguments. 

Let  there  be  given  two  formulas  F  and  G.  Suppose  that  F  represents  a 
certain  function  f,  and  also  a  representation  for  f  can  be  obtained  from  G 
by  possibly  choosing  different  variable  symbols.  Then  we  will  say  that  F 
is  equivalent  to  G  (F  =  G). 

Remarks.  Usually,  if  we  are  dealing  with  D-formu7as  for  a  single  domain 
D,  we  represent  the  identity  function  by  a  variable  symbols  (i.e.,  we  omit 
the  operator  symbol  for  the  identity).  In  the  formal  model  we  use,  we  cannot 
do  this  since  it  would  be  ambiguous.  Also,  for  purely  technical  reasons,  we 
insist  that  every  operator  has  at  least  one  argument  (otherwise,  the  wording 
of  several  definitions  and  results  would  be  more  cumbersome).  Thus,  we  do 
not  allow  constants.  Rather,  instead  of  constants,  we  use  operators  with  > *f i *. 
fictitious  argument.  Suppose  we  are  given  the  formula  F.  Occasionally,  wc 
will  say  "Replace  the  variable  x  (in  F)  by  the  constant  a".  This  is  to  be 
interpreted  as  "Replace  the  variable  x  with  a  (y)  "  where  y  is  a  variable 
symbol  not  appearing  in  F. 
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Let  f:  D  -♦  D  and  let  g  be  an  arbitrary  finite  function  of  n  arguments 
with  domain  E  =  Dn  and  such  that  g  =  f  E.  Let  F  be  a  D-formula  for  (i.e. , 
representing)  f.  Then  we  can  also  say  thi, t  (F,E)  represents  g.  From  now  on 
we  will  not  be  pedantic,  and  we  will  simply  say  that  F  represents  g.  Some  of 
the  main  results  in  this  thesis  are  concerned  with  the  question,  given  a 
specific  function  D  -»  D  ,  how  much  can  we  simplify  its  representation  if  we 
choose  an  E-formula  for  it  with  D  E. 

If  F  is  an  arbitrary  formula,  then  the  set  of  variables  appearing  in  it 
will  be  called  its  support  (denoted  by  S(F)).  The  set  of  operators  appearing 
in  F  will  be  called  its  basis  (denoted  by  B(F)). 

Let  $  C  0.  xhgn  the  get  of  formuias  p  SUch  that  B(p)  c  $  will  be  called 
the  set  of  formulas  over  $.  Hopefully  without  too  much  danger  of  ambiguity, 
we  will  also  say  that  $  is  a  basis  of  operators  (for  formulas  over  $).  All 
the  significant  results  we  will  describe  deal  with  formulas  over  $  when  $ 
is  finite  (and  representing  a  set  of  operators  with  domain  D  for  a  single 
value  of  D).  From  now  on,  whenever  a  basis  of  operators  $  is  introduced, 
it  is  always  assumed  finite.  Usually,  we  are  interested  only  in  bases  that 
allow  all  function  D  -♦  D  for  a  certain  D  and  arbitrary  n  to  be  represented. 
Such  bases  will  be  called  complete  bases  (for  D). 

Notat'.on.  Elements  of  ^  will  always  be  denoted  by  lower  case  Latin 
letters.  The  various  bases  of  operators  we  will  use  will  be  denoted  by 
capital  Greek  letters;  operators  (i.e.,  basis  elements)  will  be  denoted  by 
lower  case  Greek  letters  (except  for  well  known  operators  for  which  established 
notation  exists);  formulas  will  be  denoted  by  capital  Latin  letters;  and  D 
will  always  refer  to  the  domain  of  formulas,  d  will  denote  D  . 
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1.2.2  Example. 

If  D  =  (0,1),  then  the  functions  Dn  -»  D  for  arbitrary  n  are  known  as 
Boolean  functions.  A  complete  basis  for  (0,1)  conists  of  the  binary  operators 
A  (conjunction)  and  V  (disjunction),  and  the  unary  operator  —  (complementation). 
This  basis  shall  be  denoted  by  II.  The  formula  F  =  V(A(—  (x^  ,x2)  ,  A(x  ,—  (x  ))) 
over  n  represents  ^  P  x2  (the  mod  2  sum  of  ^  and  x2).  Usually,  this  is 
written  as  x!  A  x2  V  xl  A  x2*  We  have  s(p)  =  fx1,x2),  and  B(F)  =  n  . 

A  convenient  representation  of  formulas  is  by  trees.  This  is  a  standard 
device  that  will  not  be  described;  suffice  it  to  say  that  to  each  formula  F 
there  corresponds  a  tree  T (F)  whose  terminal  nodes  are  labelled  with  variable 
symbols  and  the  nonterminal  nodes  with  basis  symbols.  As  an  example,  le'  F 
be  as  defined  in  Example  1.2.2.  Then  T(F)  is  shown  in  Fig.  1.1. 

Given  a  formula  F,  we  need  a  notation  for  subformulas  of  F. 

The  definition  of  sub formula  is  the  standard  one:  (1)  F  is  a  subformula 
of  F,  (2)  if  F  =  cd(F1,.  . .  ,Fk) ,  then  if  Fi  for  1  ^  i  5  k  is  not  a  variable 
symbol,  any  subformula  of  is  a  subformula  of  F,  and  (3)  subformulas  of  F 
are  only  objects  satisfying  (1)  and  (2).  Subformulas  distinct  from  F  are 
proper  subformulas. 

Let  G  be  a  subformula  of  F  such  that  G  =  cpO^ , . . .  , Hg ) .  Then  we  will  say 
Hi  =G.i  for  1  5  1  5  L  This  notation  can  be  iterated.  In  Example  1.2.2, 

F.2.2  =  ~<'X2^’  However ,  note  that  F  ^  is  a  variable  symbol  which  according 
to  our  definition  is  not  a  formula.  This  can  be  remedied  by  replacing  this 
particular  occurrence  of  the  variable  symbol  x^  by  id(x^).  For  this  reason 
we  will  require  that  all  the  bases  we  consider  contain  the  identity  function 
whether  this  is  specifically  mentioned  or  not. 
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If  G  F. j (1). j (2). . . j (r) *  then  J  =  j (!)j  (2)« . . j (r)  is  called  the  index 
of  G  (for  completeness,  let  X  denote  the  index  of  F).  If  G  is  a  proper  sub¬ 
formula  of  F,  then  F  =  H(X,G)  where  X  U  S (G)  =  S(F)  and  H(X,z)  is  a  formula 
(determined  by  j)  where  z  appears  only  once.  We  write  H  =  F/G.  In  this  case, 
with  F  and  G  as  given,  we  will  also  write  S*(G)  =  X  (i.e.,  the  variables  of 
F  that  appear  outside  of  G).  We  define  S*(F)  =  .  The  subscript  F  will 

generally  be  suppressed  when  it  will  be  clear  to  what  formula  F  we  refer  to. 

In  what  follows,  whenever  we  will  deal  with  a  subformula  G  of  F,  it  will  be 
assumed  that  the  index  of  G  is  also  given;  for  if  not,  then,  e.g.,  F/G  and 
S*(G)  are  not  uniquely  defined. 

Frequently,  formulas  will  occur  where  certain  variables  have  been  re¬ 
placed  with  constants.  Suppose  F  is  a  formula  over  $,  X  =S(F),  and  a  £  D; 

then,  F  with  all  variables  except  those  in  X  replaced  by  a  will  be  denoted 
X  X 

by  F^.  Obviously,  F^  is  a  formula  over  $  U  {a}.  If  f  is  an  arbitrary  function, 
X  a  subset  of  its  arguments,  then  f  has  the  analogous  meaning,  viz.,  the 
function  obtained  from  f  by  restricting  the  elements  outside  of  X  to  a. 

The  functional  table  of  f*  is  obtained  from  that  of  f  by  deleting  all  columns 
except  those  that  correspond  to  X  and  retaining  only  the  rows  with  a 
entries  in  the  deleted  columns. 

1.3  Measures  of  Complexity 

Let  us  introduce  the  three  most  widely  studied  measures  on  formulas: 

(1)  Length.  The  length  of  a  formula  F,  denoted  by  L(F) ,  is  the  number  of 
occurrences  of  variable  symbols  in  F.  In  other  words,  it  is  the  number  of 
terminal  nodes  of  T(F). 
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(2)  Cost.  The  cost  of  a  formula  F,  denoted  by  C(F),  is  the  number  of  opera¬ 
tor  symbols  in  F.  In  other  words,  it  is  the  number  of  nonterminal  nodes  of 
T(F). 

(3)  Depth.  The  depth  of  a  formula  F,  denoted  by  D(F) ,  is  the  depth  of  nest¬ 
ing  of  operators  in  F.  In  other  words,  it  is  the  number  of  arcs  on  the  long¬ 
est  branch  of  T(F). 

Now,  given  an  arbitrary  function  f:  Dn  -*  D  and  a  (finite)  basis  $,  we 
define  the  length  of  f  over  $  as 

L(f,$)  =  min((i:  There  exists  a  formula  F  over  $  for  f  such  that 
L(F)  =  1)) 

If  f  canne*-  be  represented  by  a  formula  over  $,  we  define  L(f,$)  =  Simil¬ 
arly,  for  cost  and  depth. 

It  is  noteworthy  that  all  the  measures  above  are  closely  related.  In  fact, 

C0-L(f,$)  sc(f(})  SCi.L(ff$)  (1.3.1) 

c2-log2(L(f,*))  £  D(f,»)  *  c3* log2(L(f ,$))  (1.3.2) 

for  an  arbitrary  function  f  such  that  L(f,$),  C(f,$),  and  D(f,$)  are  finite, 
and  certain  constants  Cq,  c^,  c^,  and  c^  that  depends  on  $.  The  basis  *  is 
also  arbitrary,  except  in  the  case  of  the  right  inequality  of  (1.3.2)  where 
it  must  be  such  that  all  the  constants  and  the  function  g  (see  Lamma  1.3.1) 
may  be  represented. 

We  first  establish  the  relation  between  cost  and  length  (1.3.1). 
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Any  formula  F  over  5  can  be  built  up  from  one  which  uses  only  one  opera¬ 
tor  symbol  (an  elementary  formula)  by  successively  replacing  variable  symbols 
with  new  elementary  formulas.  If  F  does  not  contain  one-argument  operators, 
then  whenever  we  increase  the  cost  during  the  build-up  (by  adding  an  elementary 
formula  with  cost  1),  we  also  increase  the  length.  Specifically,  the  length 

increases  by  between  n  .  -1  and  n  -1  where  n  ,  and  n  are  respectively 

min  max  min  max 

the  smallest  number  larger  than  1  and  the  greatest  number  of  arguments  of  an 
operator  of  $.  This  results  in  the  estimate 

n . "IT  *L(F)  *  C (F)  5  ~  •  L(F)  (1.3.3) 

max  min 

where  c  =  1.  Suppose  F  contains  one-argument  operators.  In  other  words,  T(F) 
contains  nodes  with  hranching  factor  one.  Let  the  maximal  number  of  such  nodes 
that  occur  one  after  another  on  any  branch  of  T(F)  be  c*;  then  (1.3.3)  still 
applies  with  c  =  c*  +  1.  (1.3.1)  is  obtained  from  (1.3.3)  by  noting  that  the 

minimal  length  or  cost  representation  of  any  function  (over  the  chosen  basis 
$)  can  be  achieved  with  a  formula  where  c*  ^  dd  (the  number  of  functions  D  -♦  D). 

The  left  inequality  in  (1.3.2)  is  established  by  a  trivial  counting  argu¬ 
ment  (the  maximal  number  of  terminal  nodes  in  a  tree  with  branching  factor  ^ 

n  and  depth  d  is  nd  ).  The  right  side  requires  more  effort  (the  following 
iiia/v  max 

argument  is  due  to  R.  W.  Floyd).  We  first  state  the  following  obvious 
1.3,1  Lemma. 

Given  a  formula  F  such  that  F  =  F^ (X^ , F2 (X^ ) )  where  is  a  proper  sub¬ 
formula  of  F  and  F^  =  F/F2,  the  following  holds: 
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^  ^1  ^1  » *  *  *  »^i  ^l’^d-1^  »^2  ^2^ 

where  Ci  for  D  ^  i  ^  d-1  is  any  formula  representing  the  constants  0 . d-1 

(or,  as  we  have  remarked  previously,  the  one-argument  function  with  constant 

value),  and  F„  is  any  formula  representing  the  function  g(zn,...,z,  ,,z  = 

J  U  a*  1  d" 

"z0  if  zd  =  0;  Z1  if  zd  =  zd-l  if  zd  =  d-1. 


Let  F  be  an  arbitrary  formula  over  $,  and  let  G  be  a  proper  subformula 
of  F.  We  already  know  that  F  =  H(X,G).  The  claim  is  made  that  if  L(F)  >  1, 
G  can  be  chosen  in  such  a  way  that 

n 

L(H)-1,L(G)  <  *L(F)  (1.3.4) 

max 


where  n^  is  as  defined  previously.  (Remark:  L(H)-1  is  the  number  of 
occurrences  of  the  variables  of  S*(G)  in  H. ) 

To  find  G  use  the  following  procedure:  Start  with  F  and  proceed  to  sub¬ 
formulas  of  F.  Assume  you  are  considering  the  subformula  K.  Then  two  cases 

can  arise.  Either  among K  for  1  £  j  £  k  where  k  is  the  number  of  arguments 

•  J 

of  the  outermost  operator  of  K  there  is  one,  j',  such  that  L(K  , )  2:  a-L(F) 

•  J 

(0  <  a  <  1  will  be  determined  later  with  the  purpose  of  obtaining  the  lowest 

possible  estimate  of  L(H)-1  and  L(G)),  or  not.  In  the  first  case,  proceed  to 

K  ,  and  containue.  Otherwise,  set  G  =  K  „  where  j"  is  such  that  L(K  .„)  = 

max  (L(K  ))  and  terminate.  Before  the  procedure  terminates,  L(K)  2:  a-L(F). 
l^j^k 


Thus  ^  L(G)  <  a-L(F).  This  also  means  (l-a)*L(F)  £  L(H)-1  <  (1--^ _ ) 

max  nmax 

•  L(F)  (because  L(G)  +  L(H)-1  =  L(F)).  The*  lowest  bound  for  L(G)  and  L(H) 

is  obtained  by  setting  a  =  1 - —  ;  hence  (1.3.4) 

max 
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Now  apply  Lemma  1.3.1  with  G  replacing  F2  and  H  replacing 
depth  c,  depending  on  $.  If  the  outlined  procedure  is  applied 


Fr  f3  is 


recursively 


of 

to 


H(X,C.)  for  0  5  i  i  d-1  and  to  G,  we  obtain  in  (1.3.2)  c,  =  — -  where 

n  +1  .  3  loS2b 

5  _  max  T 

n 

max 

Note  that  unlike  the  cost-length  relationship,  the  minimal  value  of 
depth  may  not  be  achieved  by  the  same  formula  as  the  minimal  value  for  length. 

Apart  from  the  relationship  between  the  various  measures,  depth  and  cor' 
will  not  be  treated  further.  Even  though  in  what  follows  (in  this  chapter) 
many  things  hold  mutatis  mutandis  for  depth  and  cost,  most  of  the  specific 
discussion  and  the  examples  shall  be  confined  to  length. 


-1*4 — £rgblgpi$  Related  to  the  Length  Measure 

In  this  section  we  will  mention  several  questions  that  have  been  asked 
about  the  complexity  of  finite  functions,  their  status  as  of  this  writing, 
and  how  they  relate  to  the  work  to  be  described  here. 


t 

A  more  precise  expression  is  obtained  if  the  right  side  of  (1.3.2)  is  re- 
piaced  by  • log2(L(f ))+/  for  some  constant  l.  Namely,  if  we  start  out 

with  a  formula  F  and  decompose  it  according  to  (1.3.4)  and  Lemma  1.3.1,  then 
the  length  of  H(X,  C. )  and  G  is  bounded  by  ^L(F)+k  where  k  is  the  length  of  C 
After  n  applications  of  Lemma  1.3.1,  the  lengths  of  the  relevant  formulas  are 
bounded  by  ^  L(F)  +  k^  +  ...+  k^fk  ^  (L(F) )  +  -^ 


above. 


;  hence  the  figure 
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The  Problem  of  Aggregate  Length 

Let  $  be  a  complete  basis  (for  a  certain  domain  D).  The  statement  of  the 
problem  is:  What  is  the  largest  number  L(n,$)  such  that  there  exists  a  function 
f:  Dn-»  D  and  L(f,$)  =  L(n,$)? 

It  has  been  studied  by  several  authors,  and  is  now  effectively  disposed 
of.  Riordan  and  Shannon  [Ri42]  first  derived  a  lower  bound  for  L(n,n). 

Actually  they  studied  series-parallel  contact  networks,  but  the  two  models 
are  equivalent.  The  first  upper  bound  (for  the  same  model)  was  obtained  by 
Shannon  fSh49].  Krichevskii  fKr59]  derived  a  lower  bound  for  L(n,$)  for 
arbitrary  domains  and  bases,  while  Lupanov  [Lu59]  obtained  the  best  upper 
bound  for  the  general  case.  The  result  is 


0(L(n,$))  = 


logccn 


where  0(f(n))  =  g(n)  means  that  lim 

n  -»  =0 

are  two  remarks  that  are  in  order  here.  The  first  is  that 

Formulas  represent  finite  functions  efficiently;  i.e., 
the  total  number  of  formulas  (over  a  given  basis  $)  of 
length  up  to  L(n,$)  closely  matches  the  number  of  func¬ 
tions  of  n  variables. 


(1.4.1) 


is  finite  and  nonzero.  There 


The  second  is 


The  fraction  of  functions  Dn  -*  D  that  can  be  represented 
by  formulas  of  length  up  to  L(n,$)» (l-e)for  an  arbitrary 
0  <  e  £  1  approaches  zero  as  n  -*  °°. 


(1.4.2) 


(1.4.3) 


The  interested  reader  may  obtain  more  information  in  the  literature  cited 


above. 
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Obviously,  we  could  define  functions  c(n>f)  Md  0(D(J)  a„alogous  to 
L(n,»)  In  terms  of  the  cost  and  depth  measures.  In  genera  1,  such  functions 

(aggregate  complexity  functions)  can  be  defined  In  connection  with  any  model 

for  the  representation  of  functions  nn  -♦  n 

ctions  D  -*  D  and  any  measure  on  this  model  (an 

obvious  variation  of  L(„,«  Is  to  r«  the  condition  of  completeness  on  ,). 

It  should  be  noted  that  the  as^totlc  behavior  of  aggregate  complexity  functions 

remains  an  active  area  of  research.  For  references  or  the  subject,  see  Lupanov 
[Lu70] . 


The  Minimization  Problem 

Investigation  of  the  complexity  of  finite  functions  started  on  representa¬ 
tions  of  Boolean  functions  by  logical  circuits.  In  fact,  formulas  can  be 
thought  of  as  circuits  with  fan-out  one.  Thus,  the  first  problems  studied 
were  those  a  logic  designer  is  likely  to  ask:  Given  a  finite  function,  what 

is  the  minimal  circuit  (formula)  that  represents  it  (i.e.,  find  the  complexity, 
and  do  so  "effectively"). 

Unfortunately,  no  statlsfactory  solution  to  the  minimisation  problem 
exists  (for  any  measure).  This  does  not  mean  that  it  Is  impossible  to  obtain 
a  minimal  formula  for  a  given  function  f:  Dn  D;  rather  that  existing 
algorithms  are  Impractical.  Thus,  It  Is  always  possible  to  order  formulas 
according  to  length,  and  then  search  all  formulas  up  to  length  L(n,t)  for  the 

first  formula  that  represents  f;  but  since  there  are  dd"  functions  of  n  argu- 
ments  this  approach  is  absurb. 

At  the  present,  all  existing  algorithms  for  the  minimization  of  functional 
representations  employ  some  sort  of  an  exhaustive  search  (e.g. ,  the  Quine 
algorithm  for  the  minimization  of  disjunctive  form  representations  of  Boolean 
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functions),  In  fact,  there  la  reaaon  to  believe  that  a  more  efficient  method 
does  not  exist,  i.e. , 

1.4.1  Conjecture 

Any  generally  applicable  exact  minimization  procedure  is  comparable  (in 
terms  of  computational  complexity)  to  an  exhaustive  search  among  formulas. 

It  is  useful  to  consider  a  specific  machine  model.  Let  us  consider 
implementations  of  such  a  procedure  as  a  deterministic  one-tape  Turing  machine 
M$(see,  e.g. ,  Arbib  fAr69])  that  receives  as  its  input  the  dn-tuple  defining 
an  arbitrary  function  f:  Dn  ■*  D,  and  whose  output  is  the  minimal  formula  F 
(over  $)  for  f.  Conjecture  1.4.1  gives  us  that  the  computation  time  of 
M*  may  attalR  an  exponential  (in  the  length  of  the  input).  Let  us  venture 
a  more  restrictive  and  precise  version  of  Conjecture  1.4.1: 

1,4.2  Conjecture 

Let  M$  be  as  described,  m  is  the  length  of  its  input,  and  let  71(m)  be  a 

function  such  that  -♦  0  as  m  -*  *  for  an  arbitrary  constant  c  >  1.  Then 

c 

the  proportion  of  inputs  of  length  m  at  which  the  running  time  of  M  exceeds 
T|(m)  approaches  1  as  m 

Actually,  the  specific  machine  model  on  which  the  procedure  of  Conjecture 
1.4.1  above  is  implemented  is  not  particularly  important.  It  cau  be  easily 
shown  (see,  e.g.,  Arbib  TAr69] ,  Chapter  4)  that  different  deterministic  machine 
models  (this  applies  to  the  most  widely  used  models,  e.g.,  one-tape  and  multi¬ 
tape  Turing  machines)  can  simulate  each  other  in  such  a  way  that  the  running 
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time  of  one  is  related  to  the  running  time  of  another  at  most  by  a  polynomial. 
In  this  way,  whenever  the  running  time  is  exponential  (in  the  length  of  the 
input)  in  one  case,  it  must  be  sc  also  in  others. 

It  seems  that  Conjecture  1.4.1  was  first  expressed  by  Yablonskii  [Ya59]. 

A  very  interesting  result  connected  with  this  subject  was  recently  obtained 
by  Cook  [Co71].  He  obtained  strong  evidence  that  a  simpler  problem  requires 
nonpolynomial  time.  The  problem  is  that  of  recognizing  whether  a  certain 
disjunctive  normal  form  (for  a  Boolean  function)  represents  the  constant  1. 

Cook  showed  that  if  this  problem  could  be  solved  in  polynomial  time  (by  a 
deterministic  one-tape  Turing  machine),  then  a  number  of  other  problems  that 
are  regarded  as  very  difficult  (e.g.  ,  given  the  graphs  Gj^  and  G2,  determine 
whether  G^  is  isomorphic  to  a  subgraph  of  G2;  the  recognition  of  primes;  etc.), 
would  also  be  rapidly  computable.  Note  that  a  fast  minimization  machine 
would  give  us  also  a  fast  constant  recognizer;  hence,  Cook's  results  supports 
Conjecture  1.4.1. 

Ihc__.Clafi3ificatlon  Problem 

In  view  of  the  difficulty  of  finding  an  exact  nontrivial  solution  to  the 
minimization  problem  (i.e.,  one  that  does  not  employ  exhaustive  search),  present 
research  is  directed  at  establishing  bounds  for  the  length  of  functions.  We 
consider  sequences  of  functions  f^,...  of  1,...  arguments  and  study  the 
growth  rate  of  the  length  of  f^.  Thus,  we  can  talk  of  classes  of  linear 
(length)  sequences,  quadratic  (length)  sequences,  etc.  Also  of  nonpolynomial 
(length)  sequences.  Unfortunately,  if  a  sequence  belongs  to  a  nonlinear  class, 
it  is  very  difficult  to  estimate  its  length.  We  cannot  even  assign  represen¬ 
tatives  to  the  polynomial  classes  of  degree  >  2,  let  alone  the  nonpolynomial 
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classes.  In  fact,  at  the  present  we  have  only  a  very  limited  store  of  examples 
of  nonlinear  sequences. 

Consider  the  Boolean  function  f2  x^  Subbotovskaya  [Su61]  gave  a 

striking  proof  that  0(L(f2,  n))  n3/2.  It  was  known  already  to  Shannon  (see 

[Sh49] ,  or  [Ya54] )  that  0(L(f2,II))  £  n2  (the  length  of  this  sequence,  of  course; 
grows  linearly  if  ©  is  used).  Unfortunately,  it  seems  that  the  technique  of 
CSu61]  cannot  be  generalized  to  d  >  2.  Subbotovskaya 's  result  has  recently 

been  improved  by  Khrapchenko  [Kh71].  He  succeeded  in  showing  that  0(L(f2  II)) 

>  2  .  n> 
n  .  Since  this  result  employs  a  very  interesting  technique,  and  since  it 

has  not  yet  been  translated  into  English,  it  is  reproduced  in  Appendix  B. 

Neciporuk  [Ne66]  discovered  a  sequence  of  Boolean  functions  f  such  that 
2  n 

°a<fn,*))  =  7^  for  an  arbitrary  basis  *.  It  is  true  that  the  functions 
involved  in  the  Neciporuk  sequence  are  rather  "artificial"  in  that,  while 
defined  in  a  straightforward  way,  they  have  no  special  significance;  however, 
lately  Harper  and  Savage  [ Ha 7 1 ]  have  succeeded  in  applying  the  Neciporuk  tech¬ 
nique  to  a  practical  combinatorial  problem  (The  Marriage  Problem). 

Neciporuk 's  construction  is  based  on  the  following  lemma:  Let  f  be  a 
Boolean  function  of  n  arguments.  Consider  a  subset  X  of  the  arguments  of  f 
and  the  set  of  restrictions  of  f  to  X  obtained  by  setting  the  arguments  outside 
of  X  to  constants.  Let  the  number  of  su^h  restrictions  be  r.  If  1‘  is  any 
form 'la  over  a  finite  basis  $  for  f,  then  the  number  of  occurrences  of  variables 
representing  the  arguments  in  X  is  a  c*log2r  where  c  depends  on  the  basis  ? 

(for  the  proof  of  this  sec  (Ne66]  or  [Ha71]). 

The  Neciporuk  function  f^  of  n  arguments  is  then  obtained  as  follows: 

The  n  arguments  are  arranged  in  a  rectangular  array  with  dimensions  as  shown 
in  Fig.  1.2.  Each  argument  x^  is  associated  with  a  0-1  valued  m-tuplc  a.j 


19 


such  that  (1)  not  all  components  are  0,  and  (2)  if  (i,j)  ^  (k,£)  then 
a  4  a.  ..  Then  we  define 


f  =  ®  x 

n  all  i,j  ij 


©  K(a..,k) 


ii 


where  K(a^,k)  denotes  the  conjunction  of  those  arguments  whose  second 
subscript  (t)  corresponds  to  nonzero  components  of  a... 

“ij 

It  can  be  verified  that  the  number  of  restrictions  of  f  to  the  variables 

n 

of  an  arbitrary  row  (except,  perhaps,  the  last  which  may  be  imcomplete)  ob¬ 
tained  by  replacing  the  variables  of  the  other  rows  with  constants  is  2n  m. 
This  follows  from  the  fact  that  any  Boolean  function  can  be  uniquely  represen¬ 
ted  by  a  Boolean  polynomial  (see  Lamma  4.5).  Then,  by  the  lemma  above,,  the 
number  of  occurrences  of  variables  of  any  row  (except,  perhaps  the  last)  in 

any  formula  for  f  is  ^  c*(n-m);  hence,  the  length  of  f  over  $  w  c»  ~  (n-m). 

n  n  m 

n2 

In  other  words,  0 (L(cp  ,$))  *  t - for  an  arbitrary  basis  i. 

n  l°82n 

Neciporuk's  construction  may  be  viewed  as  a  solution  to  a  special  case 
of  the  following  problem  (the  problem  of  exhibiting  a  function  of  arbitrary 
length):  Given  a  basis  $  and  a  number  k  £  L(n,$),  exhibit  a  function 
f:  D  -*  D  of  length  ^  k  over  ?.  In  Neciporuk's  case  0(k)  =»  “ 


l°g2n 

Since  so  few  examples  of  functions  that  are  known  to  be  cf  large  length 
exist  (in  spite  of  (1.4.3)),  the  reader  has  no  doubt  already  gained  the 
impression  that  this  problem  too  is  very  difficult.  However,  we  again  have 
the  trivial  solution  that  consists  in  examining  formulas  in  n  variables  in 
the  order  of  their  length,  recording  the  functions  they  represent,  and 
choosing  the  first  previously  unencountered  function  represented  by  a  formula 
of  length  2  k.  In  fact,  it  is  reasonable  to  state  an  analog  of  Conjecture 


1.4.1: 
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1«4.3  Conjecture 

The  problem  of  exhibiting  a  function  of  arbitrary  length  is  comparable 
(in  terms  of  computational  complexity)  to  an  exhaustive  search  among  formulas. 

We  again  make  this  conjecture  more  precise  on  the  example  of  determin¬ 
istic  one-tape  Turing  machines. 

l.«.3.4  Conjecture 

$  is  an  arbitrary  basis.  N?  is  a  deterministic  one-iape  Turing  machine 

with  input  (n,k)  where  n  is  arbitrary  and  k  Ufn,}),  and  whose  output  is  the 

d  -tuple  describing  a  function  f  of  n  arguments  such  that  L(£,$)  *  k.  Then 

there  exists  a  constant  c  >  1  such  that  if  k  i  e.L(n,$)  for  any  0  <  esi, 

n 

the  running  time  of  N$  on  input  (n,k)  exceeds  cc  when  n  *  n(  G). 

We  can  sun  up  the  discussion  of  the  classification  problem  as  follows. 

The  problem  is  far  from  understood.  At  the  present  no  sequence  of  functions 

A 

is  known  whose  length  grows  faster  than  n  .  Isolated  examples  of  sequences 

2 

with  growth  rate  £  n  are  known,  and  present  research  is  directed  at  inven¬ 
ting  more  general  techniques  that  can  be  used  for  estimating  the  complexity 
of  whole  classes  of  sequences.  Also  techniques  have  to  be  devised  for  d  >  2. 
The  importance  of  this  will  be  discussed  below  in  Section  1.5. 


— Specker's  Theorem 

The  first  general  technique  for  proving  the  nonlinearity  of  a  large 
class  of  sequences  (of  Boolean  functions)  was  discovered  by  Soecker  fHo08], 
Let  the  basis  II  U  (x  ®  y)  be  denoted  by  E  .  Then 
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Theorem  (Speck^-I. 

If  f  is  a  Boolean  function  of  n  arguments,  if  L(f,E)  s  c*n  for  s  ,me 

constant  c,  then  for  any  integer  m,  if  n  *  TL(m,c),  a  subset  X  =  (x, ,...,x  } 

5>  I’m 

of  the  arguments  of  f  can  be  found  such  that  (1) 

X  m  m 

f0  “  c0®  C1  *  n  (l@x  )^c?  e  X  (1.5.1) 

i-I  1  L  i-1  1 

where  Cq,  c^,  C2  are  Boolean  constants  and  ^g(m,c)  is  a  certain  number- 
theoretic  function.  Furthermore,  (2)  if  the  basis  is  II  (the  other  assump¬ 
tions  remaining  unchanged),  then  c2  **  0. 


This  theorem  has  been  used  by  Hodes  and  Specker  to  show  that  the  predi¬ 
cate 


n 

E  x  =  0  mod  k  (1.5.2) 

i-1  1 

for  k  >  2  and  x^  P  (0,1)  is  of  nonlinear  length  over  E. 

Using  the  second  statement  of  the  theorem,  they  are  also  able  to  give 

n 

an  alternative  proof  of  the  nonlinearity  of  the  length  of  ©  x  over  II. 

i-1  1 

Another  result  obtained  with  Specker 's  Theorem  is  the  fact  that  some 
geometrical  predicates  (in  particular,  connectivity)  discussed  by  Minsky 
and  Papert  [>1169]  are  of  nonlinear  length  over  E  (see  Hodes  [Ho70]). 

In  Chapter  Two  we  will  formulate  and  prove  a  generalization  of  Specker 's 
Theorem  (Theorem  2.2.2)  to  Include  the  case  d  >  2  and  multi-argument  oper¬ 
ators  in  ?.  Our  proof  reveals  the  nature  of  both  results  more  clearly. 


Will  be  discussed  in  Chapter  Two. 
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They  belong  to  a  cla93  of  combinatorial  results  reminiscent  of  Ramsey's 
Theorem  (see  Ryser  [ Ry 6 3 ] ) .  In  fact,  an  earlier  version  of  our  proof 
of  Theorem  2.2.2  used  Ramsey's  Theorem.  Besides  this,  Theorem  2.2.2 
enables  us  to  derive  the  nonlinearity  of  new  functions  (sequences  of 
functions)  such  as  counting  mod  p  where  p  is  a  prime,  d  possibly  equals 
p,  but  there  are  restrictions  on  the  basis,  etc.  An  example  of  an  im¬ 
provement  over  existing  results  is  the  connectivity  predicate.  Hodes 
[Ho70]  proves  that  it  is  nonlinear  if  d  =  2.  However,  in  Automata  Theory, 
for  example,  the  result  that  a  certain  language  can  be  computed  in  non¬ 
linear  time  if  k  states  are  used  in  the  finite  control  would  be  considered 
weak.  Rather  we  search  for  proofs  that  work  for  arbitrary  finite  controls. 
The  Generalized  Specker  Theorem  (Theorem  2.2.2)  gives  us  a  tool  for  proving 
the  nonlinearity  of  the  length  of  the  connectivity  predicate  regardless  of 
the  domain  L  and  basis  ?.  We  can  apply  it  to  connectivity  by  "reducing" 
connectivity  (for  the  meaning  of  "reduction"  see  [Mi69]  or  3.2)  to  certain 
symmetric  functions. 

We  should  note  that  the  generalization  of  Specker 's  Theorem  that  we 

prove  is  the  obvious  one  to  attempt;  but,  as  the  reader  will  see,  the  proof 

turns  out  to  be  less  straightforward.  As  an  indication,  consider  (1.5.2). 

It  does  not  generalize  directly  to  d  >  2  since,  e.g. ,  the  function  (0,1)  n-» 

(0,1)  defined  by  E  x.  =  0  mod  6  can  be  represented  in  linear  length  with 
i=l 

d  =  3.  This  is  because 

n  n  n 

fix  -  0  mod  6]  =  [  T.  x  -  0  mod  3]  A  [  t  x  =  0  mod  2) 
i=l  1  i=l  1  i=l  1 
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Hodes  and  Specker  do  not  derive  any  bounds  for  the  lengths  of  the 
functions  investigated  by  them.  This  question  is  asked  (and  to  an  extent 
answered)  in  3.3. 

1.6  Cyclic  Perceptrons 

Cyclic  Perceptrons  will  be  treated  in  Chapter  Four.  They  are  an 
application  of  ideas  of  Minsky  and  Papert  to  the  representation  of  functions 
by  combinations  of  finite  operators.  In  particular,  one  of  the  concerns 
in  [Mi69]  is  to  formalize  the  intuitive  idea  that  the  connectivity  predicate, 
being  "global"  in  nature,  cannot  be  computed  (or  represented)  by  a  "simple" 
combination  of  "local"  predicates. 

The  perceptron  is  the  predicate 

I  a  *cp  ^  o 
i€I 

where  I  is  an  indexing  set,  a±  €  Q,  the  rationals,  cp  6  $,  a  set  of  Boolean 
functions  (whose  value  is  interpreted  as  being  either  the  rational  0  or  1. 

The  cyclic  perceptron  is  defined  as 

r  a  €  Y 
tfl 

where  a^  €  F,  a  finite  field,  Y  ^  F,  and  other  symbols  have  the  same  inter¬ 
pretation  as  before.  Thus,  both  represent  a  certain  Boolean  function. 

Minsky  and  Papert  introduce  the  concept  of  the  order  of  a  perceptron 
(the  maximal  number  of  arguments  on  which  depends  where  i  ranges  over  I). 
They  define  then  the  order  of  a  predicate  as  the  minimal  order  of  a  per¬ 
ceptron  that  represents  the  predicate.  They  formalize  "local"  by  defining 
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an  infinite  predicate  sequence  to  be  local  if  and  only  if  every  member  is 
representable  by  a  perceptron  of  order  £  r,  for  some  finite  r.  They  are 
then  able  to  show  that  connectivity  is  nonlocal. 

The  concept  of  order  can  also  be  applied  to  cyclic  perceptrons.  Chapter 
Four  will  contain  results  on  the  order  of  the  various  predicates  introduced 
in  [Mi69] .  In  particular,  connectivity  is  shown  to  be  nonlocal.  This  will 
be  an  extension  (to  finite  fields  of  arbitrary  characteristic)  of  the  results 
described  in  [Vi70]. 

Chapter  Five  describes  a  model  of  computation  (Pattern  Counting  Machines) 
that  again  performs  a  "local"  computation  followed  by  a  "global"  computation. 
In  this  case  the  "local"  computation  is  even  more  constrained  than  in  the 
case  of  perceptrons.  The  result  is  that  no  matter  how  cleverly  we  utilize 
the  "local"  information  in  the  subsequent  "global"  phase,  the  connectivity 
predicated  cannot  be  computed. 


m 


T(F)  for  the  formula  F  in  Example  1.2.2 
Fig.  1.1 


column 
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CHAPTER  TWO 

A  GENERALIZATION  OF  A  THEOREM  OF  SPECKER 
2.1  e-Complexes 

Throughout  this  section,  all  formulas  are  D-formulas  for  some  fixed  (but 
arbitrary)  domain  D,  and  all  operators  are  functions  Dr  -♦  D. 

Given  the  formulas  F^...^,  we  shall  call  the  formula  F  «©(F  ,...,F  ) 
where  cp  is  an  arbitrary  operator  a  parallel  combination  (PC)  of  F. , . . .  ,F  .  cp  is 
called  the  decoding  operator  of  F. 

Let  F (X , z )  be  a  formula  where  the  distinguished  variable  z  appears  only 
once,  and  let  G  be  an  arbitrary  formula.  Then  F(X,G)  shall  be  called  a  series 
combination  (SC)  of  F  and  G  through  z. 

2.1.1  Definition 

We  give  an  inductive  definition  of  an  elongated  n-component  ^-component) 
for  n  S:  0. 

(1)  Let  cpin  be  an  arbitrary  unary  operator  and  z  an  arbitrary  variable  symbol 

Then  cpin(z)  is  an  e0-component.  z  is  the  input  variable  while  cp  is  the 

in 

input  operator. 

(2)  Let  cp  be  an  arbitrary  binary  operator,  G  an  arbitrary  e^-component , 
and  x  (  S(G).  Then  F  -  tf(x,G)  (or  cp(G,x))  is  an  en"component.  The  input  vari¬ 
able  and  input  operator  of  G  are  also  the  input  variable  and  input  operator 

of  F.  x  is  a  lateral  variable  of  F.  Any  lateral  variable  of  G  is  also  a 
lateral  variable  of  F.  cp  will  be  called  an  Internal  operator  of  F. 
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An  example  of  an  en-component  is  given  in  Fig.  2.1.  Let  F  be  an  arbitrary 
e^-component ,  and  let  x  be  the  sequence  of  lateral  variables  arranged  in  the 
order  they  are  connected  to  the  branch  of  T(F)  extending  to  the  input  variable. 
Then  x  is  the  late ra  1  sequence  of  F.  If  F  is  an  e^-component ,  then  the  lateral 
sequence  of  F  is  \  (the  empty  sequence).  For  example,  the  lateral  sequence  of 

the  e-component  in  Fig.  2.1  is  x,,...,x  . 

I  n 

An  en-component  with  all  internal  operators  equal  is  a  homogeneous 
e^ -component . 

2.1.2  Definition 

A  formula  F  is  an  e^-complex  if  (1)  F  is  a  PC  of  the  e^components  Fj . 

Fr>  and  (2)  the  lateral  sequence  of  Ft  for  2  £  i  «:  r  is  either  equal  to  the 
lateral  sequence  of  F^ ,  or  the  reverse  of  it. 

Fl»**‘*  are  the  components  of  F.  If  the  variables  of  Fj  are  numbered 
as  in  Fig.  2.1,  the  second  condition  of  Definition  2.1.2  means  that  any  compo- 
nent  Fi . Fr  either  appears  as  in  Fig.  2.1,  or  as  in  Fig.  2.2.  The  compo¬ 

nents  of  the  former  kind  will  be  known  as  standard  components,  while  those  of 
the  latter  kind  will  be  called  the  reverse  components.  The  lateral  sequence 
of  Fj  will  also  be  called  the  lateral  sequence  of  F. 

Both  in  the  case  of  e^components  and  e^-complexes ,  one  or  both  indices 
will  occasionally  be  omitted  if  the  particular  property  they  refer  to  is 
irrelevant  to  the  argument  at  hand. 

An  e-complex  composed  of  homogeneous  e-componcnts  Lj  a  homogeneous 
c-complex. 
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One  might  wonder  what  the  purpose  of  introducing  e-complexes  is  since 

for  appropriate  r  and  m  every  function  of  n  variables  can  be  represented  by 
r 

an  em-complex.  Thus,  it  would  seem  that  this  class  of  formulas  is  trivial. 

However,  we  will  be  concerned  with  e^-complexes  where  r  remains  fixed  as  n 

grows  without  bounds,  and  this  will  allow  us  to  obtain  interesting  results. 

We  introduce  some  notation.  Let  F  be  an  e^-component  with  lateral 

sequence  (2) *  *  *  * ,xi  (n)'  a  €  D  is  an  arbitrary  constant,  cp^  denotes 

the  internal  operator  corresponding  to  xJ/lN.  Also  set  cp  .  =  cp,  .  Then 

i  ( j  )  n+1  in 


<Pj(a’<Pj+l(a»**-*Vl(a'CPk)-“))  if  1  *  J  <  k  *  n+1 


i 


(j,k) 


<Pj  if  Uj  «ks  n+1 
undefined  otherwise 


Note  that  9*^  ^  is  a  unary  operator  if  j  ■  n+1,  otherwise  it  is  a  binary 
operator  (if  it  is  defined  at  all).  Usually  we  will  suppress  the  superscript 
a  because  it  will  be  clear  what  constant  is  referred  to. 

We  now  state  the  simple 


2.1.3  Proposition 

Let  F  be  an  e^-component  with  lateral  sequence  x  and  input  variable  z. 

Y  is  an  arbitrary  subsequence  of  x  of  length  m  ^  0  and  a  €  D  is  an  arbitrary 

constant.  If  we  denote  the  set  consisting  of  z  and  the  elements  of  ^  by  Y, 

Y 

then  F  is  equivalent  to  an  e  -component  G. 

3  m 
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Proof 

Let  x  «  (x1,x2,...,xn)  and  1  =  (x1(1)t  *1(2)’“  •  »*i(in))  £  *•  Set  i(0)  =  0 
and  i  (m+1)  =  n+l.  Then  G  has  the  operators  ^  =  cp^1(j_1j+1  for  i  <:  j  <: 

1,1+1  ls  the  input  operator  of  G).  □ 

2.1.4  Remark 

Obviously,  Proposition  2.1.3  holds  for  e-complexes  as  well;  one  merely 
has  to  perform  the  above  construction  for  each  component. 

Proposition  2.1.3  will  be  frequently  invoked.  Namely,  we  will  take  an 
e-complex  F,  select  a  subsequence  jSx,  the  lateral  sequence  of  F,  and  obtain 
G  as  above.  In  this  case,  G  is  called  the  result  of  an  a -merger  with  basis 
y  on  F. 

We  introduce  another  restricted  class  of  formulas. 

2.1.5  Definition 

A  series  parallel  combination  of  e-components  (SPCeC)  is  obtained  accor¬ 
ding  to  the  following  rules: 

(1)  An  e-component  is  an  SPCeC. 

(2)  Let  F  and  G  be  an  e-component  and  an  arbitrary  SPCeC  respectively. 
Then  the  SC  of  F  and  G  through  the  input  variable  of  F  is  an  SPCeC. 

(3)  If  F1,...,Fr  are  SPCeC's,  then  a  PC  of  Flt...,F  is  an  SPCeC. 

(4)  An  SPCeC  is  only  an  object  satisfying  (1),  (2),  or  (3). 
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Given  an  arbitrary  SPCeC  F,  we  describe  its  set  of  components.  If  F 
consists  of  the  single  e-component  G,  then  G  is  the  only  component  of  F. 

If  F  is  the  SC  of  an  e-component  G  and  another  SPCeC  H,  then  the  set  of 
components  of  F  consists  of  G  and  the  set  of  components  of  H.  If  F  is  a 
PC  of  F1,...,Fr,  then  the  set  of  components  of  Fconsists  of  the  sets  of 
components  of  F£  for  1  £  i  £  r.  Among  the  components  of  F,  those  whose  input 
variable  corresponds  to  a  terminal  node  of  T(F)  will  be  called  terminal 
components  while  the  others  will  be  called  internal  components.  An  example 
of  an  SPCeC  is  given  in  Fig.  2.3.  This  particular  SPCeC  has  four  terminal 
components  and  two  internal  components. 

2.1.6  Proposition 

An  SPCeC  is  equivalent  to  a  PC  of  r  e-components  where  r  =  d*I+J  and  I 
and  J  respectively  are  the  number  of  internal  and  the  number  of  terminal  component  of  F. 

Proof  F  can  be  converted  into  a  PC  of  e-components  by  using  Lemma  1.3.1. 

The  estimate  of  the  number  of  e-components  in  the  PC  is  also  obtained  from 
there.  p] 

Remark  It  is  a  simple  matter  to  verify  that  if  F  of  Proposition  2.1.6 
has  k  components,  then  I  £  k-1;  and  thus  r  £  d* (k-l)+l. 

2.1.7  Proposition 

F  is  a  SPCeC  with  k  components  F. ,. . .  ,F.  .  F,  for  1  ^  i  i  k  is  an  e  ■ 

l  k  i  n 

component  for  n  S  0,  and,  furthermore,  the  sets  of  lateral  variables  of  F 

i 

and  Fj  are  equal  for  1  <:  i,  j  rJ  k.  Let  X  be  the  set  of  lateral  variables 
of  F^  and  let  Z  be  the  set  of  input  variables  of  F.  Then  for  any  m  ^  0  and 
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a  c  D  it  u  i  ^(m.k)  ‘there  T^(m,k)  is  a  certain  function  (to  be  defined), 

there  exists  a  subset  7  C  X  with  M  ■«  m  such  that  is  equivalent  to  an 

e  -complex  G  with  Y  as  the  set  of  lateral  variables.  Furthermore,  r  £  d*(k-l)+l. 
m 

Proof  If  m  *  0,  we  can  inroediately  apply  Proposition  2.1,6  and  obtain  an 

r 

eQ-complex  where  r  is  as  described  in  the  statement  of  the  proposition;  thus 
Tl^(0,k)  =  0.  We  assume,  therefore,  that  m  >  0. 

We  recall  the  following  familiar  result: 

2 

Let  i(l),  i (2) , . . . ,i ( (p-1)  +1)  he  a  sequence  of  distinct  integers.  Then 

we  can  extract  a  subsequence  of  length  p  that  is  either  increasing  or  decreasing 

(for  the  proof  see  Bergc  [Be71]  p.  16). 

Without  loss  of  generality,  we  can  assume  that  the  lateral  sequence  of 

is  Xj,...,xn.  Then  the  lateral  sequence  of  F2  is  x^^,  (2) ’ ' ' ' ,xi (n)* 

The  sequence  i(l),  i(2) .  i(n)  consists  of  distinct  integers;  therefore, 

2 

if  n  J  (n^-1)  +1,  we  can  apply  the  above  result  snd  find  a  subset  X^  C  x  of 
n^  variables  such  that  after  performing  an  a-merger  with  basis  X^  on  all 
components  of  F,  the  lateral  sequences  of  the  descendants  of  Fj  and  Fj  are 
either  the  same  or  opposite.  We  can  continue  in  this  way,  processing  one 
after  another  all  components.  We  end  up  with  an  SPCeC  with  components 
Gp...,G|(  such  that  the  lateral  sequence  of  G^  for  2  s  i  S  k  is  either  equal 
to  that  of  Gj  or  the  reverse  of  it.  To  obtain  G,  we  apply  Proposition  2.1.6. 

In  order  that  *y|  =  m,  we  must  have 

2k-! 

n  2  (m,k)  =  (m-1) 

for  m  ^  1.  The  estimate  for  r  is  obtained  from  the  Remark  following 


Propos i t ion  2.1.6. 


32 


Another  equivalence  that  will  be  used  later  is  given  by 
2,1,8  Lenina 

E  is  an  e^-complex,  X  is  the  set  of  its  lateral  variables,  and  Z  is  r-he 
set  of  its  input  variables.  Then  for  any  ns  *  0  and  a  €  D,  if  n  2  T^On.r), 
there  exists  a  subset  Vex  with  |  a  m  such  that  E^^  is  equivalent  to  a 
homogeneous  er-complex  F. 


Proof  If  m  »  0,  we  simply  use  Proposition  2.1.3,  and  the  result  is  a  homo¬ 
geneous  eg-complex  (obviously,  any  Cg-complex  is  homogeneous).  Then  T^(0,r)  =0. 
Thus,  from  now  on  we  assume  that  m  5s  1. 

The  proof  will  be  given  for  the  special  case  when  E  has  two  components: 
a  standard  component  E^  and  a  reverse  component  E^.  It  will  then  only  be 
Indicated  how  to  generalize  the  proof. 

A  procedure  (The  Homogenizing  Procedure--HP)  will  be  described  that  will 
2 

transform  an  e^-complex  G  consisting  of  a  standard  component  and  a  reverse 
component  G2  with  p  ^  V^(q)  (f°r  a  function  Tl^  that  will  be  defined  later) 
and  with  the  properties:  (1)  There  exist  (possibly  empty)  subsets  Rj  and 
r2  c  D  such  that  o^a^jNj  -  idR  (identity  on  Rj)  and  ’^(a  ,y)  f  R2  =  idR 
for  1  ^  i  S  p  where  and  is  an  operator  of  G^  and  G2  respectively 
and  the  first  argument  corresponds  to  the  lateral  variable,  and  (2) 

"(1  Si,  j  «p)  [oi(x,y)  ?  rx  =  C2j(x,y)  =  ^(x.y)!  (2.1.1) 

(i.e.,  the  operators  of  G ^  are  identical  on  the  inverse  image  of  R^).  Simil¬ 


arly  for  the  operators  of  G2  on  R2> 
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Remark  Note  that  if  R^  and  R2  include  the  range  of  every  operator* 
Property  2  translates  into  the  identity  of  the  operators.  In  particular, 
this  holds  if  Rj  =  R2  *  D. 

2 

Remark  Note  that  an  arbitrary  e  -complex  satisfies  Properties  1  and  2 
with  R1  »  R2  ■  <t>. 

2 

The  result  of  applying  HP  will  be  an  e  -complex  H  that  will  either  be 

q 

homogeneous,  or  will  have  Properties  1  and  2  with  S1  and  S2  replacing  Rj  and 
R2  respectively  and  R^  or  R2  ^  S2»  Due  to  the  Remarks  above  and  to 

the  fact  that  D  is  finite,  repeated  application  of  HP  on  E  finally  yields  F. 
The  condition  on  n  is 

n  *  T12fa,2)  ■  l)3(l)3(...n3(m)...))  (2.1.2) 

2d  times 

This  bound  for  n  corresponds  to  the  worst  case  when  R^  or  R2  increase  by  only 
1  on  each  application  of  HP. 

Before  describing  HP,  note  the  useful  fact  that  because  of  Property  1, 
Properties  1  and  2  are  preserved  under  a-mergers. 

Description  of  HP  The  lateral  sequence  of  G  is  of  length  (v+l)»u-l  for 
certain  values  of  u  and  v  that  will  be  defined  later. 

Consider  the  sequence 

fl  £1 

(m((k-l)*u+l,k-u)’  *  ((v-D-u+1,  (v-k+l).u)) 

for  k  =  l,...,v.  Sequence  (2.1.3)  is  illustrated  in  Fig.  2.4.  The  two 
vertical  lines  represent  and  G2;  the  numbered  horizontal  outlets  represent 
the  lateral  variables  (with  the  corresponding  number);  the  boxes  indicate  the 
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variables  and  operators  that  take  part  in  the  formation  of  any  particular 

®(i,j)  and  *(i,j)’  30  be8ide  a  variable  indicates  that  it  is  not  set  to 
the  constant  while  'a'  indicates  that  it  is  set  to  a;  the  two  checked  boxes 
represent  the  first  member  of  (2.1.3). 

In  the  sequence  (2.1.3),  either  (Case  I)  the  ranges  of  cp 

((k-l)*u+l,k*u) 

and  ^((v-k^u+l,  (v-k+l)*u)  ^0r  1  ^  k  ^  v  are  included  in  and  respectively, 
or  (Case  II)  not. 

Case_I.  If  v  is  large  enough,  we  can  find  q  identical  elements  in  the 

sequence  (2.1.3).  Let  the  indices  k  corresponding  to  these  elements  be 

k]_ » •  •  •  ,kq.  Performing  an  a-merger  with  this  sot  as  basis,  the  desired 

e-complex  H  is  obtained.  Note  that  in  this  case  we  use  Property  1  of  G. 

Namely  if  cp*  is  the  first  component  of  a  pair  in  (2.1.3)  whose  range  C  Rj 

and  if  <P  i?  an  arbitrary  operator  of  Gj,  then  9(a,cp*)  =  cp*  (similarly  for 

the  second  components  of  the  pairs  in  (2.1.3)  and  operators  of  G2).  Thus, 

the  components  of  the  identical  pairs  in  (2.1.3)  become  the  operators  of  H. 

d2  2  d2 

A  bound  for  v  is  q*  (d  )  (d  is  the  number  of  operators  D2  •*  D). 

Case-^I.  Assume  ^((X-D-u+i ,  X.u)(b»c)  *-  R1  for  some  b>  c  6  D  and 

1  S  1  S  V  (thC  case  ”hen  *((v-i).„+l,  *  S  «"  bc  wctcd 

similarly).  But  then 

«-u)(b,c)  *  R1  (2-1.4) 

for  all  0  S  j  s  u-1  (as  a  consequence  of  Property  1).  Provided  that  u  is 
large  enough,  we  can  find  an  element  e  £  D  that  appears  w  times  in  the  se¬ 
quence  (2.1.4).  Let  the  indices  j  corresponding  to  the  appearances  of  e  he 
j (1 ),..., j (w)  (see  Fig.  2.5).  Obviously  then  (all  the  variables  considered 
except  x^  have  been  set  to  a), 
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®oe*u-j(t),  *‘u-j(t-i)-i)(a,e)  =  e 

for  2  £  t  £  w  (the  first  argument  of  cp^  ^  corresponds  to  the  lateral 
variable).  Thus, 


W(X.u-j(t),  X-u-j(t-l)-l)(a,y)f  R1  U  =  id 
At  this  point  we  consider  separately  two  cases: 

Case  I la  ^  =  1  and  j(w)  =  u-1.  We  perform  an  a-merger  with  the  basis 

consisting  of  the  variables  with  indices  u-j(t)-l  for  1  £  t  £  w-1.  As  a  re- 

2 

suit  of  this  we  obtain  an  e  , -complex  G'  such  that  Property  1  holds  for  G,' 

W"  I  1 

on  Rx  U  f  e)  (G^  satisfies  Property  1  on  c  R2;  in  any  event  R2~R2  =  6). 

Note  that  at  this  point  Property  2  is  still  satisfied  only  on  R,  and  R0  by 
Gj  and  G2  respectively. 

Case  lib.  X  4  1  or  j (w)  <  u-1.  We  perform  an  a-merger  with  the  basis 

consisting  of  the  variables  with  indices  X.u-j(t)-l  for  1  £  t  £  w.  As  a  result 

2 

of  this  we  obtain  an  e  “complex  G"  such  that  Property  (almost)  holds  for  G 

w  1 

(the  same  remarks  regarding  G^'  and  Property  1  as  well  as  Gj',  G”  and  Property 

2  apply  as  in  Case  Ila).  The  only  exception  may  be  cp”  (the  operator  of  Gj' 

that  is  closest  to  the  decoding  function).  By  definition,  CP"  =  to.,  .  .  .  .  ... 

1  (l,X*u- j(w)-l)’ 

and  there  is  no  assurance  that  cp"(a,e)  =  e.  We  may  rectify  this  situation 

by  absorbing  cp^'  into  the  decoding  function  in  the  following  way:  Let  G"  = 

Q(G”,  G").  Now  set  x^  =  a  (after  the  a-merger  the  variables  have  been 

renumbered).  Let  S(G")  “  { x.l  =  U.  Then  we  have  (C,r)  U  =  G'  -  6(cp"(a ,  C'  ) ,  C’.  ' 

l  all 

where  G^  equals  G|  minus  cp|  and  G2  equals  G 2  with  the  input  operator  modified 

as  follows:  i!r!  =  il'"(a,\[’"  )  (remember  that  G"  is  a  reverse  components,  and, 
in  w  in  2  r  >  > 

2 

hence,  x.  is  attached  to  ty").  Clearly,  G1  is  an  e  -complex  satisfying 
1  w  w-1 
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Property  1  with  Rj  U  (e)  replacing 

We  can  now  resume  considering  Cases  Ila  and  b  together.  To  obtain  H 
(with  components  and  H2)  we  must  find  among  c for  1  £  i  £  W-1  q  operators 
that  are  identical  on  the  inverse  image  of  R^  U  (e]  (i.e.,  (2.1.1)  with 
Rj.  U  (e)  replacing  R,)  and  again  perform  an  a-merger.  We  again  emphasize 
that  the  operators  of  H2  are  identical  only  on  the  inverse  image  of  R2>  and 
this  property  has  not  been  violated  by  any  of  the  transformations  of  the 
original  e-complex  G. 

To  obtain  q  operators  that  are  identical  on  the  inverse  image  of  R  U  fe] 

d2  a2  1 

it  is  sufficient  that  w«l  ^  q*d  ;  therefore,  u  ^  d.  (q.dd  +1)  and 


fL(q)  -  q  *d*  6^+q*d  •  (®  +S)5d-1 


(2.1.5) 


.2 

where  6  =  d  .  This  is  obtained  from  the  values 
Recall  that  ^(q)  *  (v+l)*u-l,  the  length  of  G. 
obtained  from  (2.1.5)  and  (2.1.2). 


of  u  and  v  derived  above. 
T)2  for  r  =  2  can  then  be 


The  proof  for  the  general  case  is  obtained  by  defining  the  Generalized 
H  Procedure  (GHP)  with  the  corresponding  function  We  consider  instead 

of  (2.1.3)  the  sequence 


(CD' 


1  r'  1  r" 

(s,t)’***  ,CD(s,t)’  *  (s'  ,t') . ♦(s'.t)5 


where  s  =  (k-l)-u+l,  t  =  k-u,  s'  =  (v-k).U+l,  and  t'  =  (v-k+l).u.  co* , . . .  ,cd[  ' 

denote  the  operators  of  the  standard  components  of  G  while  . ^  denote 

the  operators  of  the  reverse  components  (r '  +  r"  =  r).  Without  detailed 
argument  we  state  that  in  the  general  case  v  =  q. 6r  while  u  remains  the  same 
(u  is  determined  by  the  requirements  of  Case  II  at  which  time  only  one 
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component  is  considered).  From  this  we  obtain  analogously  to  (2.1.5) 

Vq)  =  q2*d*6r+1+q*d‘  (6r+6)+d-l 

T^Cnijr)  can  then  be  obtained  from 

n2(m,r)  =  ^4(T14(...n4(m)...» 

^ - — V - ' 

r*d  times 

which  is  an  analog  of  (2.1.2).  ^ 

2.1.9  Remark 

As  we  have  seen,  TL,  in  Lemma  2.1.8  depends  on  r,  the  number  of  components 
of  F.  However,  „e  shall  mostly  be  using  e-complexes  that  contain  many  components 
that  are  identical  except  for  the  input  operator  (such  e-complexes  are  obtained 
e.g.,  by  the  use  of  Proposition  2.1.7).  It  may  be  checked  that  in  the  application 
of  GHP  only  one  representative  from  each  such  group  of  components  need  be 
considered.  This  significantly  reduces  11.  Similarly,  in  computing  ^  from 
(2.1.5),  only  /• d  compositions  are  required  where  i  is  the  number  of  groups  of 
similar  components  (corresponding  to  d  compositions  for  each  group  of  similar 
components ). 

2,1,10  Remark 

The  operators  of  a  homogeneous  e^-complex  F  obtained  as  a  result  of 
applying  Lenina  2.1,8  possess  an  added  property  that  will  be  used  later:  Let 
R  be  the  range  of  <P(x,y)  an  internal  operator  of  R  (x  corresponds  to  a  lateral 
variable);  then  (a,y)  R  =  idR  (the  identity  on  R).  This  fact  follows  from 
the  definition  of  HP  (GHP).  This  particular  property  of  the  operators  of 
the  components  of  F  will  be  called  the  IR-propertv  relative  to  y.  In  what 
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follows,  we  will  always  suppress  "relative  to  y"  since  there  is  no  danger 
of  ambiguity.  We  will  simiarly  suppress  the  subscript  R  unless  we  will  be 
interested  in  a  specific  range.  We  will  abbreviate  "F  is  an  e-component 
(complex)  whose  operators  possess  the  la-property"  to  "F  is  an  e-component 
(complex)  with  the  I  -property". 

A  familiar  and  convenient  way  of  representing  a  binary  operator  (£>(x,y) 

is  by  a  labeled  directed  graph.  The  graph  of  CD,  denoted  by  F(‘d),  is  defined 

as  follows:  The  nodes  of  F(cd)  are  labeled  with  elements  of  D.  A  directed 

arc  labeled  with  a  6  D  exists  from  b  to  c  if  and  only  if  cp(a,b)  =  c. 

If  D  =  ( 1,  2,  3,  4]  and  R  =  (2,  3,  4}  an  example  of  a  graph  F(cd)  for 

2 

an  operator  cp  with  the  IR-property  is  shown  in  Fig.  2.6. 

Given  an  arbitrary  e^component  F,  the  output  of  the  operator  cpR  is 

\  (\  'Vl  (\+l . “V  <  W  (y)  > ' ' '  >  > ; 

in  the  case  of  a  homogeneous  e^-component  with  internal  operator  CD  this  will 
be  abbreviated  to 

®(Vk+i—V  ”i„(y>)- 

2.2  The  Generalized  Specker's  Theorem 
We  first  give  the  following 

2.2.1  Definition 

Let  $  be  an  arbitrary  basis  and  a  ^  D  any  constant.  Let  F  be  a  formula 

over  $  with  S(F)  =  X  U  Y  U  Z  such  that  | X I  ^  n  -1  (the  maximal  number  of 

max 

arguments  of  an  operator  of  $) ,  Y  is  disjoint  from  X,  but  otherwise  arbitrary, 
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and  Z  =  (z)  is  a  singleton  (disjoint  from  X  and  Y)  such  that  z  occurs  only 
once  in  F.  The  set  of  functions  f(X,z)  represented  by  all  possible  such 
formulas  F  with  the  elements  of  Y  replaced  by  the  constant  operator  a  will 
be  denoted  by  $  . 

In  particular,  every  operator  of  $  with  all  but  k  ^  1  arguments  (k  is 

arbitrary)  replaced  by  a  is  in  ?a.  Note  that  if  to(X,z)  f  $a,  then  o  may 

qualify  for  $  by  virtue  of  a  number  of  different  representations.  If  z 

(or  any  variable  of  X)  in  any  one  of  them  corresponds  to  a  variable  that 

occurs  only  once,  it  is  called  a  distinguished  argument ).  The  other  arguments 

are  called  free  arguments.  Thus  we  may  easily  find  a  basis  $  and  an  operator 

cp  such  that  all  arguments  are  at  the  same  time  distinguished  and  free. 

We  now  define  a  restricted  class  of  e-components  and  c-complcxes:  9  is  an 

arbitrary  basis  and  a  (  D  is  any  constant.  Let  F  be  an  arbitrary  e^- component ; 

d  a 

then  F  is  an  eQ-component  over  9  .  Let  cpfx,z)  r:  9  be  a  binary  operator,  7. 
a  distinguished  argument  (hence  x  is  free),  and  G  an  c^-component  over  *n; 
then  cd(x,G)  is  an  e^-component  over  ?a.  An  e-complex  over  $a  is  an  o-complcx 
such  that  all  its  components  are  e-components  over  ?a. 

The  main  result  of  this  chapter  is 

2.2.2  Theorem 

Let  there  be  given  the  function  f:  D°  -*  D  such  that  L(f,?)  •£  c.n  for  some 
constant  c  and  basis  $.  Then  for  any  m  2  1  and  a  c  D  if  n  ^  ^  (c  ,m),  tliere 
exists  a  subset  Y  of  the  arguments  of  f  such  that  |y|  =  m  and  f^  is  either  a 

tl 

r  ii 

constant  or  is  represented  by  F,  a  homogeneous  e^-complcx  over  ?a  with  the  I  -properl  \, 
Y  as  the  set  of  lateral  variables,  constant  input  operators,  and  r  ^  d-(2c-l)+l. 
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Proof 

If  f  has  m  fictitious  arguments,  then  let  Y  be  the  set  of  these  arguments 
Y 

and  f^  is  a  constant.  From  now  on  we  assume  that  f  has  £  m-1  fictitious 
argumentt. 

The  statement  of  the  theorem  gives  us  that  theie  exists  a  formula  E 

over  $  such  that  L(E)  £  c*n.  Therefore,  there  are  >-  1/2. n  variable  symbols 

representing  the  arguments  of  f  which  either  do  not  appear  in  E  or  appear 

s  2*c  times.  In  other  words,  there  are  *  l/2n-m+l  variable  symbols  t^at 

actually  appear  in  E  and  such  that  the  number  of  occurrences  of  each  is  ^  2*c. 

Denote  the  set  of  these  variables  by  X^. 

^  n  ^  g(n2»^c^tn-I|  we  can  apply  Lemma  A. 9  and  obtain  a  subset 

c  I  I  X2 

X2  X1  wlth  !X?I  "  n?  and  suc^  that  E  is  equivalent  to  E  ,  and  SPCeC  over 
$  with  at  most  2c  components  and  such  that  the  set  of  lateral  variables  of 
every  component  is  X2> 

If  n2  >  ^(n-j  »2c)  we  can  apply  Proposition  2.1.7  and  obtain  n  subset 
X3  **  X2  1^!  “  °3  such  that  Eq^  is  equivalent  to  Eg,  an  e^  -complex 

over  $a  with  r  £  d,(2c-l)+l).  The  estimate  for  r  is  obtained  at  this  point. 

If  ng  *  'r12(m,2c).  we  can  apply  Leona  2.1.8  and  Remark  2.1.9  ar.d  obtain 
F,  the  desired  homogeneous  e^-complex  over  ?a  with  the  Ia-property.  The 
Ia  property  is  a  consequence  of  Lentna  2.1.8. 
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Discussion  of  T)|j.  The  present  proof  yields 

115(tn,c)  -  2.1^0^  (T)2(m,2c),2c),2c)-ta-l 

The  exact  representation  of  1^  Is  extremely  complex,  and  In  what  follows  wc 
shall  use  only  a  very  rough  approximation.  In  Appendix  A  It  Is  seen  that 
Vt,k)  (as  a  function  of  k)  grows  faster  than  iexp(b,2k)  for  any  constant  b. 
The  functions  T|^  and  contribute  only  Insignificantly  to  this,  and  thus  we 
s  tate: 


^  lexp(b,4c)  for  c  *  c(b) 
and  an  arbitrary  constant  b 


(2.2.1) 


(Later  we  shall  se  that  the  size  of  1)^  prevents  us  from  obtaining  any 
Interesting  bounds  for  the  functions  Investigated  with  Theorem  2.2.2.  The  size 
of  g  which  contributes  uost  to  1}^  results  from  the  technique  used  In  Lemma 
A. 3  to  obtain  a  nesting  sequence  for  a  given  formula  F.  It  Is  not  known 
whether  this  technique  can  be  improved.  Our  guess  Is  that  It  cannot  be.)  □ 


.2._3  On  decker's  Theorem 

In  this  section  It  will  be  shown  how  Specker's  Theorem  follows  from 
Theorem  2.2.2  (the  statement  of  Specker's  Theorem  Is  given  In  section  1.5). 

In  Theorem  2.2.2  set  D  *  (0,  1],  f  «  y,  a  «  0  and  let  f  be  as  described. 
Then  by  Theorem  2.2.2,  we  obtain  that  for  an  appropriate  choice  of  n,  we  can 
find  a  subset  X  of  the  arguments  of  f  with  |x|  -  m  and  such  that  I*  is  either 
a  constant  or  represented  by 

♦<Flt...,Fr) 
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where  1]  (0,  1]  and  for  1  i  i  ^  r  is  either  a  standard  or  reverse 

homogeneous  e^-component  over  £  with  the  I  “property  and  constant  input 
operators.  The  value  of  r  is  bounded  as  described  in  Theorem  2.2.2. 

We  now  analyze  the  various  functions  that  can  be  represented  by  e-compo¬ 
nents  with  these  restrictions.  First  note  that  £°  consists  of  all  Boolean 
binary  operators;  furthermore,  if  f(x,z)  £  1°,  then  both  x  and  z  are  free 
(because  every  Boolean  binary  operator  can  be  represented  over  £  in  such  a 
way  that  each  variable  appears  only  once),  and  thus  there  are  no  restrictions 
on  the  use  of  operators  in  the  e-components  we  encounter. 

All  possible  graphs  T((o)  for  ep  c  £°  are  shown  in  Fig.  2.7.  The  ones  that 
satisfy  the  I°-property  are  starred.  The  functions  obtained  by  choosing  a 
value  for  the  constant  input  operator  for  the  starred  graphs  are  shown  in 

Table  2.1.  In  general,  this  function  is  either  bQ  ^  b  •  tt  where  tt  =  n  (1  ^  x  ) 

m  i=l  i 

or  Cq  c^a  where  a  =  ^  Xj^  for  some  values  of  bQ,  b^  or  cQ,  c^.  Now,  taking 

into  consideration  the  fact  that  tt*o  =  0,  and  that  every  i|r:  (0,  l]h  -*-0,  1) 

can  be  uniquely  expressed  as  a  Boolean  polynomial 

2h-l 

0*  c  *M 
i=0  1  1 

where  c^  c  (0,  1]  and  is  the  monomial  (of  degree  one  in  each  variable)  in 
those  among  x^,...,x^  corresponding  to  nonzero  bits  of  the  binary  representation 
of  i (see  Lemma  4.5)  we  obtain  the  first  part  of  Specker's  Theorem. 

The  second  part  of  Specker's  Theorem  could  be  obtained  directly  at  this 
point;  however,  we  will  derive  a  generalization  of  it  in  Example  3.1.3,  and  thus 


omit  it  here. 
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It  mast  be  pointed  out  that  our  derivation  of  Specker's  Theorem  results 
in  a  slightly  larger  bound  for  n;  however,  since  no  known  application  requires 
a  specific  value  for  the  bound,  this  is  immaterial.  Specker's  bound  (see 
[H068])  is  obtained  from  the  function 

(i(m,0)  =  m 

fk+iN6kn(m,k-D 

|i(m,k)  =  4'K  '  *|i(|i(m,k-l),k-l) 

by  setting  T|^(m,c)  =  2p(m,2c).  Our  bound  is  slightly  larger  due  to  the  addi¬ 
tional  processing  implicit  in  the  application  of  Proposition  2.1.7  and  Lemma 
2.1.8.  However,  p  resembles  T)^  and  this  function  by  far  contributes  the  most 
to  T)^ ;  thus,  we  can  state  that  the  bounds  are  approximately  equal. 

Finally,  let  us  note  the  fact  that  Theorem  2,2,2  allows  us  immediately 
to  amplify  Specker's  Theorem.  Namely,  the  statement  of  the  theorem  involves 
the  basis  consisting  of  all  binary  Boolean  operators.  However,  the  proof 
of  Theorem  2,2,2  works  for  bases  consisting  of  operators  of  an  arbitrary 
number  of  arguments. 
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F  Is  an  SPCeC 
Fig.  2.3 


r  i1 


46 


(k-1) . u+1 


k»u-l 

k«u 


( v~ 1 ) • u+1 


I  I  I 
I  •  • 
•  •  • 


•  !  i 


v*u+l 


(k+l).u-l 


k.u+1 

k*u 


•  *  | 

•  !  '•  )  Y 


v.u-1 


(v+1) .  u- 1 


x  =  a 


Y  =  V 


((k-l)«u+l,k.u) 


((v-k) .u+1, (v-k+1) ,u) 

Illustration  of  the  HP  procedure  (1) 
Fig.  2.4 
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F  C 


Illustration  of  the  HP  procedure  (II) 
Fig.  2.5 


2.  3 


,3,4} 


Fig.  2.6 
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i 

Table  of  functions  that  can  be  represented  by  e-components 
with  the  I  -property  (see  Fig.  2.7)  and  constant  input  oper¬ 
ators  if  D  »  (0,  1} 


Table  2.1 
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CHAPTER  THREE 


APPLICATIONS  OF  THE  GENERALIZED  SPECKER  THEOREM 

The  principal  results  obtained  previously  [H068,  Ho70]  by  the  use  of 
Specker's  Theorem  are 


3.0.1 

r 

A  new  proof  that  the  Boolean  function  .0.  x,  is  of  nonlinear  length  over 

t  r' 

FI  .  This  is  accomplished  as  follows.  First  note  that  the  restriction  of  the 

mod  2  sum  of  n  variables  obtained  by  setting  certain  variables  to  0  is  again 

a  mod  2  sum  (but  of  a  smaller  number  of  variables).  Now  apply  Specker's 

n 

Theorem  (see  1.5).  Suppose  ^0^  x^  is  of  linear  length  over  IT.  Choose  n 
large  enough  to  obtain  m  =  3.  The  theorem  states  that  for  this  particular 
bases  c2  =  0  in  (1.5.1).  However,  it  can  be  checked  that  in  this  case  no 
choice  of  Cq  and  c^  will  yield  the  mod  2  sum  of  three  variables.  A  contra¬ 
diction. 

3.0.2 

n  n 

The  function  f:  (0,  1)  -»  {0,  1}  defined  by  f  =  1  i  f  and  only  if  T.  x.  0 

i  =  1  1 

mod  3  is  of  nonlinear  length  over  E.  We  proceed  similarly  as  before.  Assume 
it  is  of  linear  length.  Apply  Specker's  Theorem  with  n  sufficiently  large  to 


Of  course  the  results  of  Subbotovskaya  and  Khrapchenko  are  stronger  ior  this 
particular  example. 
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obtain  m  =  3.  If  we  replace  x^,  X2,  x^  in  (1.5.1)  once  by  1,  0,  0,  and 
another  time  by  1,  1,  1,  then  the  value  of  (1.5.1)  remains  unchanged.  However 
the  value  of  f  (with  all  variables  except  x^,  x^,  x^  replaced  by  the 
constant  0)  is  different  on  these  two  assignments.  Again  a  contradiction. 

Both  of  these  results  were  derived  by  Hodes  and  Specker  in  [H068]. 

We  might  note  that  the  technique  of  3.0.2  can  easily  be  generalized  to 
counting  mod  k  where  k  is  an  arbitrary  integer  (see  (1,5.2)). 


3.0.3 

Certain  geometric  predicates  (see  [Mi69]),  in  particular  the  connectivity 
predicate,  are  of  nonlinear  length  if  expressed  with  binary  Boolean  operators 
(this  result  was  obtained  by  Hodes  in  f H07 0] ) .  We  will  not  discuss  this  in 
greater  detail  now  since  this  technique  will  be  treated  later  in  3.2, 

In  this  chapter  we  will  use  Theorem  2.2.2  to  generalize  all  these  results 


3. 1  Counting  mod  p 

Consider  the  function  (0,  1}°  -*  [0,  11 


f£<xl . *n) 


1  if  T.  x  =  0  mod  p 
i=l 

0  otherwise 


then 


3.1.1  Theorem 


If  p  is  a  pri)-.-’,  if  |d|  <  p,  then  fj^  is  not  of  linear  length  over  an 


arbitrary  basis. 
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Proof 

Suppose  the  statement  of  the  theorem  is  not  true.  That  is,  there  exists 

a  prime  p,  a  finite  set  D  such  that  |d|  <  p,  a  basis  C  of  operators  on  D,  and 

L(fj^,  i)  £  c*n  for  some  constant  c. 

First  note  that  if  X  is  a  subset  of  the  arguments  of  fp  and  Ixl  *  m.  then 

(Vo  =  fm*  We  can  now  aPP1y  Theorem  2.2.2.  For  an  arbitrary  m,  if  n  is 

sufficiently  large,  there  exists  a  subset  X  of  the  arguments  of  fp  with  I X I  *  m 

n  ii' 

D  X 

and  such  that  (fn)Q  is  represented  by  a  homogeneous  em-complex  F  (over  $°  and 
with  the  I°-property)  with  X  as  the  set  of  lateral  variables.  In  addition, 
since  fjj  is  a  Boolean  function,  the  lateral  variables  of  F  are  restricted  to 

Co,  l}. 

Consider  now  any  component  F£  of  F  and  where  is  the  internal 

operator  of  F£.  Since  Cp£  has  the  I°-property,  T (cp^ )  has  the  general  appearance 

of  Fig,  3.1.  F ^  is  determined  by  co^  and  the  constant  input  operator  a^.  Now 

let  m  ^  d  and  consider  the  sequence  (Sj )  for  0  a  J  S  m  where  sQ  ■  ai  and 

8J  *  cP1<H...ll,ai)  for  j  >  0.  Let  sR^  be  the  first  element  in  the  sequence 
J  times 

that  is  repeated  at  some  later  point;  in  fact,  let  k(i)  be  the  position  of  the 

first  occurrence  of  this  element.  Let  k(i)+i(i)  be  the  position  of  the  second 

occurrence  of  this  same  element.  Then  we  shall  call  k(i)  the  prefix  of  Ft 

while  X(i)  will  be  called  the  period  of  F^. 

Clearly,  if  F,  is  a  standard  (reverse)  component,  then  co. (x,x„. . .x  , 

L  *  i  1  2  m-k 

ll...l,a^)  Cw^(xmxjn>t^...x^^ll,..l>a^))  where  k  *  k(i)  is  a  function  of  the 
number  of  l's  among  x^  x2,...,xm-R  (x^ , . . .  .xj  mod  i(i). 
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Thus, 

If  we  set  Y  »  +!»•••»*„*  1  and  choose  k^kp  to 

exceed  or  equal  the  prefixes  of  all  the  reverse 

(standard)  components  of  F,  then  represents  a 

function  of  the  number  of  l's  among  the  variables  of 

Y  mod  /cm(A(l) , . . .  ,.£(r)).  (3.1.1) 

Y 

On  the  other  hand,  by  the  initial  assumption,  F*  is  a  function  of  the  number 
of  1  s  among  the  variables  of  Y  mod  p;  this  results  in  a  contradiction  since 
d  <  p. 

On  the  basis  of  (3.1.1)  we  can  obtain  the  following 
3,1,2  Theorem 

Let  D  be  an  arbitrary  domain,  #  is  a  certain  basis,  and  p  is  an  arbitrary 
integer  >  1.  If  is  such  that  any  e-component  over  5^  with  the  I^-property 
and  constant  input  operator  has  period  one,  then  fjj  is  of  nonlinear  length 
over  $. 


3.1,3  Example 

This  is  an  example  of  a  basis  satisfying  the  hypothesis  of  Theorem  3.1.2. 
Consider  an  arbitrary  domain  D  ■  (0,1,. . . ,d-l} .  Then  a  complete  basis 
for  D  is  »  (min(x,y)  ,  max(x,y)  ,0, 1, . . .  ,d-l ,  eQ(x) , . . . ,  e^  (x)}  where  min 
and  max  are  defined  in  the  usual  way,  0,...,d-l  are  the  constants,  and 

(  d-1  if  x  a  t 

ei(x>  ■  i 

l  0  otherwise 
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(Note  that  *  ( A, V, 0,1,-, id) ;  thus,  a  result  on  the  nonlinearity  of  the 

length  of  a  certain  function  over  ^  is  also  a  result  on  the  nonlinearity 

of  the  same  function  over  IT;  in  particular,  applying  Theorem  3.1.2  to  , 

(0,1' 

we  obtain  (2)  of  Specker's  Theorem.) 

*8  interesting  because  it  gives  rise  to  an  analog  of  the  disjunctive 
normal  form  for  arbitrary  D:  Consider  the  table  for  an  arbitrary  function 
f:  Dn  -*  D.  Then 


> 


dn-l 

f  =  max  (M. ) 
i-0  1 


ft. 

where  equals  0  if  the  current  assignment  is  not  the  i*  assignment  and  the 

tH 

value  of  the  function  at  the  i  assignment  otherwise.  is  represented  as 
follows 


Mi  =  tnin^Ca(i ,  1)  ^xl^ . ea(i,n)(xn),fi) 

where  a(i,j)  is  the  jth  component  of  the  i^  assignment. 

We  claim  that  satisfies  the  hypothesis  of  Theorem  3.1.2.  Note  that 
fl  b 

f°r  all  a,  b  £  D  because  tD  contains  all  the  constants.  Therefore, 

•Jf 

we  will  write  simply  ^ 

*  0 

Given  (0  €  jfc  with  the  I  -property,  the  statement  that  there  exists  b  f  0 
such  that  the  homogeneous  e-component  with  internal  operator  f  and  b  for  its 
input  operator  has  period  l  is  equivalent  to  saying  that  there  exists  a  subset 
L  c  D  with  | L 1  =  l  and  w(0,z)  L  is  the  identity  (id  )  while  co(l,z)  L  is  the 

L 

permutation  with  cycle  length  &  (p^). 

* 

We  contend  that  for  any  L  £  D,  cp(x,z)  and  c,  e  6  D,  if  tp(c,z)  L 
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and  ®(e,z)  L  are  1-1,  then  ®(c,z)  L  -  <p(e,z)  L.  Since  id^  i*  if  t  >  1, 

this  will  establish  the  original  claim. 

This  can  be  proved  by  induction  on  the  depth  6^  of  the  distinguished 

variable  z  in  the  formula  F  that  represents  to  (since  there  may  be  many  such 

formulas,  let  F  be  one  of  the  formulas  where  the  depth  of  z  is  minimal. 

If  6  ■  1  then  either  F  »  max(F',z),  or  F  >  min(F*',z).  Assume  the  first 

* 

case  (the  second  can  be  argued  similarly).  By  definition  of  jfc_,  F  contains 
only  the  variable  x.  If  we  replace  x  by  c ,  F'  represents  a  constant  c '  c  D. 

Now  if  c'  £  L,  then  cp(c,z)  1.  is  the  identity,  otherwise  it  is  not  1-1. 

If  6^  >  1,  then  either  co(x,z)  =  eA  (o'  (x,z))  where  tl)'  t  and  6^,  <  6^ 

ff 

or  cp(x,z)  «  cp*(x,m'"  (x,z))  where  to”,  <p”'  C  £D  and  fc^„,  6^,,,  <  6^  (to  see  this, 
think  of  F).  In  any  case  tp' ,  to",  and  to”1  satisfy  the  inductive  hypothesis, 
and  we  are  done. 

Note  that  Theorem  3.1.1  and  3.1.2  hold  with  fP  replaced  by  the  function 

gp:  Dn  -♦  (0,1)  given  by  E  x.  ■  0  mod  p  since  gP  f  (0,1)°  «  fp. 
n  1  n 

3.2  Connectivity 

The  connectivity  predicate  was  already  discussed  in  1.6.  It  attacted 
considerable  attention  after  Minsky  and  Papert  [Mi69]  succeeded  in  obtaining 
interesting  results  on  the  complexity  of  perceptrons  that  represent  the 
connectivity  predicate.  Works  that  follows  [M169]  and  that  treat  specifically 
the  representation  of  the  connectivity  predicate  by  finite  operators  are,  e.g. , 
[ HO 70]  ,  [ML7U,  and  [Vi70]. 
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Minsky  and  Papert  describe  a  circuit  for  computing  the  connectivity 

predicate  of  depth  (of  the  order  of)  (log2n)2  which  on  intuitive  grounds 

seems  minimal.  This  circuit  translates  into  a  formula  of  nonpolynomial  length. 

Thus,  the  connectivity  predicate  seems  to  be  a  good  benchmark  for  testing 

estimation  methods  for  the  complexity  of  functions  (i.e.,  any  appropriarely 

general  method  which  is  presumed  able  to  give  estimates  for  length  up  to 
los2n 

f(n)  <  n  should  declare  the  connectivity  predicate  complex). 

Consider  a  set  of  n2  variables  (x^)  for  1  s  i,  j  s  n;  then  the  connec¬ 
tivity  predicate  is  the  function  cn:  {0,  l)n  (0,  1}  defined  as  follows: 

(we  will  not  give  a  formal  definition  since  the  formalization  is  obvious) 

Given  a  specific  assignment  to  the  variables,  consider  it  as  a  square  array 
of  0's  and  l's.  Then  cn  -  1  on  the  empty  pattern  (i.e.,  consisting  of  all  0's), 
or  if  the  l's  form  a  connected  pattern.  By  "connected"  we  mean  that  any  two 
l's  can  be  linked  by  a  sequence  of  adjacent  l's  (two  l's,  corresponding  to  the 
variables  x^  and  xR/  are  adjacent  if  li-k!  +  lj-j|  -  1).  For  example,  the 
pattern  in  Fig.  3.2  is  connected. 

The  general  approach  used  here  to  obtain  an  estimate  for  the  length  of 

c_  Is  to  consider  reductions  of  c  . 
n  n 

Given  an  arbitrary  function  f,  a  function  g  will  be  called  a  k-r eduction 
of  f  if  g  is  obtained  from  f  by  replacing  each  argument  of  f  by  a  function  with 
at  most  k  arguments. 

Suppose  we  want  to  prove  that  f£  of  i  =  1,2,...  arguments  is  of  nonlinear 
length  (over  the  basis  $).  Assume  there  exists  a  k-reduction  g^  of  f^  such 
that  the  number  of  arguments  m  in  gR  is  m  *  Ct.n  for  some  constant  0  <  a  £  1. 
Assume  Ufn,$)  s  c-n  for  some  constant  c.  That  is,  there  exists  a  formula 
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Fn  f0r  fn  and  L(V  *  C#n*  Since  the  length  of  any  function  of  k  arguments 
is  bounded  by  L(k,$),  we  obtain 


L(gn,*)  <  c*L(k,®)*n 


by  making  substitutions  for  variables  in  F  . 

n 

If  fg1)  Is  rearranged  (and  renumbered)  in  the  order  of  increasing  number 
of  arguments,  and  all  but  one  functions  with  the  same  number  of  arguments 
are  deleted,  then  we  obtain 

L(g^,$)  ^  c 1 »m 


for  some  constant  c'.  Finally,  if  we  can  prove  (e.g.,  by  applying  Theorem 
2.2.2)  that  g^  is  nonlinear,  we  obtain  a  contradiction,  and  we  are  done. 

Modes  [Ho70]  shows  the  nonlinearity  of  the  length  of  cr  over  T.  by  reducing 
cr  to  the  function 


m 

V  (A  y ) Ay 
i-1  j»<i  J  1 

(i.e.,  exactly  one  variable  is  1)  which  can  then  be  proved  to  be  nonlinear 
using  Specker  8  'Hieorem.  Unfortunately,  this  reduction  does  not  work  for 
domains  with  more  than  two  elements  because  this  function  is  linear  over  an 


appropriate  basis  in  such  domains.  However,  another  approach  works,  and  we 
can  state 


3.2.1  Theorem 

Regardless  of  the  size  of  D  and  the  nature  of  $,  c  is  of  nonlinear 

n 

2 

length  (i.e.,  L(cn>$)  £  c«n  is  not  true  for  any  D,  $,  and  constant  c). 
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Proof 

Minsky  and  Papert  [M169]  succeed  In  reducing  c^  to  counting  mod  2  by 
exhibiting  a  contact  network  such  that  its  connectivity  depends  on  the  number 
mod  2  of  contact  variables  equal  to  1,  and  then  by  simulating  this  network 
on  the  square  array  of  variables  (they  call  it  the  "retina'*).  We  shall  proceed 
similarly. 

ct  is  reduced  to  the  function  sjj:  (0,  l}n  -»  (0,  1)  (for  an  appropriate 
t)  defined  as  follows: 

1  if  ^  arguments  are  equal  to  1 

Sn  = 

0  otherwise 


s^,  can  be  represented  by  the  connectivity  of  a  contact  network  is 

shown  in  Fig.  3.3a.  It  has  p  contact  arm3  for  each  variable  y^.  The  0  value 

of  y^  corresponds  to  the  upward  position  of  the  corresponding  arms  while  the 

1  value  of  y^  corresponds  to  the  downward  position.  The  contact  arm  of 

are  arranged  in  p  rows  (n  arms  in  each  row).  Whenever  an  arm  for  y^  is  in 

the  upward  position,  it  is  connected  to  an  arm  for  y^+^  in  the  same  row;  if 

the  arm  for  y^  is  in  the  downward  position,  it  is  connected  to  an  arm  for 

y^+^  in  the  next  row.  Thus,  it  may  be  easily  checked,  point  is  connected 

to  B  if  and  only  if  at  least  p  among  y, . .  are  1.  It  may  also  be 

p+1  °  1  n 

verified  that  in  this  case  all  the  contact  arms  in  the  network  are  connected 


either  to  An  or  to  B  ,,  and  since  these  two  are  connected  together,  the  whole 
U  p+1 

network  is  connected. 

S*3  in  turn  can  be  simulated  by  a  rectangular  array  of  0's  and  l's 
n  n 

where  certain  positions  are  constant  and  others  depend  on  the  y^'s  (see 
Fig.  3.3b).  The  size  of  is  (3(p+l)+l) . (3n+p-l). 
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We  now  show  that  sP  is  a  1-reduction  of  cfc  for  some  t.  This  is  done 
by  cutting  RP  into  smaller  rectangular  pieces  along  vertical  lines.  The 
first  piece  is  of  length  (/-l)*q  where  q  =  3(p+l)+l  and  l  will  be  defined 
later,  the  second  through  f-lst  piece  is  of  length  (*-2).q,  while  the  /th, 
last,  piece  is  of  length  between  1  and  (jt-l)*q.  These  pieces  are  then 
arranged  into  an  L'  q  x  4*q  square  pattern  T^  as  shown  in  Fig.  3. 4  (the 
arrangement  depends  on  the  parity  of  £).  Corresponding  rows  of  adjacent 
pieces  are  connected  by  3-  or  C-  shaped  patterns  of  0's  (in  the  case  when 
one  of  the  positions  ulong  the  cut  at  the  row  in  question  is  0)  or  l's. 

The  unused  positions  of  TP  (corresponding  to  the  case  when  the  last  piece 
is  not  of  the  maximal  length)  are  replaced  by  0's. 

t  is  set  to  £•  q  and  the  variable  in  cf  is  replaced  by  the  correspon¬ 
ding  position  in  TP  (one  among  0,  1,  yt ,  or  yJ,).  Obviously,  the  function 

obtained  by  this  replacement  is  sp.  If  1  i  ^  £  ia  obtained  as  ^xl  where 

n  q 

x  is  the  positive  solution  of 

2  „ ,  3n+o-l 

x  -2  (x-1)  =  - K — 

9 

We  also  have  that  n  e»(l/3q)t,  and,  thus  (by  the  reasoning  outline  previously), 

if  c_  is  linear  so  is  sP. 
t  n 

With  the  assumption  that  sP  is  linear,  we  apply  Theorem  2.2.2  with  a  =  0. 

In  this  way  we  obtain  that  (sP)q  where  Z  is  a  certain  subset  of  the  arguments 
of  sP  of  size  m  is  represented  by  an  em_complex  F  (with  the  requisite  restric¬ 
tions)  where  m  is  arbitrary.  Note  that  (sP)if  =  sp. 

n  0  m 
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As  noted  in  3.1,  if  •  sufficiently  large  number  of  variables  at  the 
beginning  and  end  of  the  lateral  sequence  of  F  is  replaced  by  1,  then  F  with 
this  substitution  represents  a  function  of  the  number  of  l's  among  the 
remaining  variables  mod  the  1cm  of  a  set  of  integers  £  d.  The  number  of 
variables  that  have  to  be  set  of  1  is  u  S  2(d-l)  (at  most  d-1  at  each  end 
of  the  lateral  sequence).  Thus  we  obtain  a  representation  of  the  function 
8m-u  if  p  a  u*  If  P  *  2 (d- 1)4-2,  we  obtain  a  function  sj  for  i  *  2.  However, 

it  is  clear  that  Sj  is  not  a  function  of  the  number  of  l's  mod  k  for  any  integer 

k.  Thus,  we  have  arrived  at  a  contradiction,  and,  hence,  c  is  of  nonlinear 

n 

length  over  any  basis  ?.  0 

l. 3  The  Length  of  Symmetric  Functions* 

As  we  have  seen  in  the  previous  examples,  Theorem  2.2.2  has  been  applied 
only  to  functions  that  are  either  symmetric  or  that  can  be  reduced  to  symmetric 
functions.  While  we  know  of  no  formal  statement  that  can  be  proved  and  that 
asserts  that  this  indeed  exhausts  the  applicability  of  Theorem  2.2.2,  it 
intuitively  seems  probable. 

In  this  section  we  will  discuss  several  bounds  on  the  length  of  symmetric 
functions  (both  specific  functions  and  all  symmetric  functions).  Recall  that 
in  1.4  we  have  already  mentioned  several  such  bounds  (Subbotovskayi,  Khrapchenko). 


All  of  the  results  in  this  section  were  suggested  by  A.  R.  Meyer. 
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Does  Theorem  2.2.2  (or  Specker's  Theorem)  give  us  any  information  on  the 
length  of  the  functions  Investigated?  Hodes  and  Specker  do  not  treat  this 
subject,  and,  In  fact,  the  bound  that  can  be  obtained  Is  very  weak;  however, 
we  do  mention  It  for  the  sake  of  completeness. 

In  an  application  of  Theorem  2.2.2  (or  Specker's  Theorem)  to  a  certain 
function  f,  we  proceed  with  the  assumption  that  L(f,$)  £  c*n.  To  apply 
Theorem  2.2.2  we  must  have  n  ^  7]^(m,c)  where  m  is  a  sufficiently  large  number 
to  obtain  a  contradiction.  However,  m  does  not  depend  on  c.  Thus,  n  depends 
only  on  c  and  m  Is  assumed  constant. 

Consider  now  c  as  a  function  of  n.  We  ask  what  Is  the  maximal  value 
c  for  c(n)  for  which  Theorem  2.2.2  can  be  applied  (and  a  statement  contradic¬ 
ting  L(f,$)  JS  c*n  obtained).  c(n)*n  Is  then  a  lower  bound  for  L(f,$).  Due 
to  (2.2.1)  c  grows  slower  than  (l/4)*ht  (p,n)  where  ht  (p,n)  =  maximal  x 
such  that  n  ^  lexp  (p,x).  Then  we  have 

c(n)*n  s  (l/4)*ht  (p,n)*n  (3.3.1) 

for  an  arbitrary  constant  b  >  1  and  for  sufficiently  large  n. 

This  bound  seems  unrealistically  low,  and  it  is  useful  to  compare  it 

3 

with  known  bounds  for  length  for  the  particular  function  f^  ove’’  some  bases 
consisting  of  Boolean  operators  (we  will  suppress  the  subscript  n). 

3 

It  has  already  been  established  that  f  is  of  nonlinear  length  if  D  = 

3  3  0  3  1 

(0,  1}  (see  3.1).  We  introduce  the  following  notation:  f  =  f  ’  ,  f  ’  , 

3  2  n 

and  f  ’  stand  for  I  x  0,  1,  and  2  mod  3  respectively.  We  will  repre- 

i-1  1 

sent  f^’0,  f"*’*,  f^’^  by  the  formulas  F^,  F^,  and  F^  respectively.  F^  is 
obtained  by  the  following  recursive  relation 
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F°(x)  =  F°(Y)  A  F°(Z)  V  F1(y)  A  F2(Z)  V  F2(Y)  A  F1(Z)  (3.3.2) 

0 

(If  X  Is  the  singleton  (x),  then  F  (X)  =  x) 

L(F° (X))  =  L(F°(Y))+L(F°(Z))+L(F1(Y))+L(F2(Z))+L(F2(Y))+L(F1(Z)) 

1  2 

Similar  identities  can  be  obtained  for  F  (X)  and  F  (X).  When  these  identities 
are  used  recursively,  we  obtain 

log  6 

0(L(f  ,$))  ^  n  »n  ,D  (3.3.3) 

an  exact  description  how  we  obtain  a  bound  of  the  form  (3.3.3)  from  a  recur¬ 
sive  relation  similar  to  (3.3.2),  see  [Ya54]. 

This  upper  bound  can  be  further  reduced  by  using  multiargument  operators. 

Ji  R  3 

Let  y :  { 0,1,2]  *♦  (0,1,2)  be  the  operator  £  y  mod  3.  Then  r  can  be 

i=l 

represented  by  a  formula  G  which  uses  y  recursively  (i.e.,  the  arguments  of 

3 

f  are  repeatedly  divided  by  l  together  with  an  outermost  decoding  operator 
(0,1,2)  *♦  (0,1).  G  is  of  linear  length.  If  we  use  D  =  (0,1),  y  can  be  en¬ 
coded  by  two  operators  y'  and  y" ,  and  G  translates  into  a  formula  such  that 

log  n  1  +  log  k 

0(L(G))  -  (2k)  -  n  £  (3.3.4) 

3 

Thus,  as  i  increases,  the  upper  bound  for  L(f  ,$)  (where  y  $)  approaches 
c*n.  However,  the  gap  between  this  bound  and  (3.3.1)  is  still  huge.  But, 
the  important  thing  to  note  is  that  any  theorem  that  retains  the  same  broad 
assumption  (bases  with  an  arbitrary  number  of  operators)  as  Theorem  2.2.2 

3 

cannot  yield  a  better  bound  for  f  than  (3.3.4). 
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Another  example  of  a  function  that  is  nonlinear  in  length  (over  all 

Boolean  binary  operators)  by  Specker's  Theorem  is  f4.  However,  it  too  has 

a  relatively  short  representation  (the  previous  and  this  example  show  that 

Theorem  2.2.2  is  a  sensitive  tool  for  deriving  the  nonlinearity  of  functions; 

i.e.,  it  can  be  used  on  functions  that  are  only  "slightly"  nonlinear). 

4 

A  representation  for  f  with  Boolean  operators  is  obtained  by  dividing 
4 

the  arguments  of  f  into  disjoint  (nonempty)  pieces  Y  and  Z,  and  adding  the 
,  .  .44 

bineary  representations  of  f  (Y)  and  f  (Z).  Let  the  binary  representations  of 

4 

f  (X)  be  given  by  the  formulas  F'(X)  and  F"(X),  obtained  by  the  following 
recursive  relations 

F”(X)  =  F"(Y)  a  F'*(Z) 

F'(X)  =  F'(Y)  ©  F'(Z)  ©  F**(Y)  A  F"(Z) 

(If  X  is  a  singleton  F"(x)  =  x  and  F'(x)  =  0) 

Consequently, 

L(F"(X) )  =  L(F"(Y) )+L(F"(Z) ) 

L(F'(X))  =  L(F'(Y))+L(F' (Z)+L(F"(Y))+L(F'(Z)) 

By  choosing  Y  and  Z  always  as  equal  as  possible,  we  obtain 

L(F"(X) )  =  n 
0(L(F'(X)))  =  n-log2n 

4  _ _  _ 

Since  f  (X)  is  represented  by  F1 (X)  A  F"(X)  n*log2n  is  also  a  bound  for 

0(L(f  5)  where  ©,  A  $. 
n 

We  now  turn  our  attention  to  an  upper  bound  for  the  length  of  all 


symmetric  functions. 
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Note  that  a  symmetric  function  g:  Dn  D  where  D  *  -0,1 . d-1]  depends 

exclusively  on  where  is  the  number  of  variables  equal  to  i. 

It  can  be  represented,  e.g.,  as 

g  =  max(M  ) 

n(l),,,»  n(d-l) 

where  n(d-l)  e<Iua^8  'I'  if  =  n(i)  and  i3  0  otherwise.  The  number 

of  combinations  of  n(l) ,. . .  ,n(d-l)  is  a  polynomial  in  n  —  (“t*1?1)  —  and  the 

d- 1 

max  function  can  also  be  represented  in  polynomial  length,  regardless  of  the 

basis  $.  The  latter  fact  is  established  by  representing  max  using  the  two- 

argument  max  recursively.  Thus,  if  M  ..  were  polynomial,  g  would 

n(i; , . . .  ,n(.d-l; 

also  be. 

M.  J.  Fischer  and  A.  R.  Meyer  discovered  that  M  .  can. 

Indeed,  be  represented  in  polynomial  length  by  using  a  special  code  for 
Integers  described  by  Avlzlenls  [Av69], 

We  will  illustrate  the  construction  on  Boolean  symmetric  functions.  It 
will  be  seen  that  if  the  basis  of  operators  is  appropriately  chosen,  the 
length  of  an  arbitrary  symmetric  function  is  bounded  above  by  a  polynomial 
of  a  surprisingly  low  degree. 

The  Avlzlenls  code  is  a  redundant  positional  representation  of  integers 
to  an  arbitrary  base  b  >  2.  We  describe  it  for  b  =*  3. 

An  integer  n  is  represented  by  all  possible  Tlog^n"!  -  tuples 

ariog3ni’--*’ai 

where  a^^  €  ( -2, -1,0, 1,2]  for  1  £  i  £  Tiog^nl  and 
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flog^nl 

E 

i*l 


n 


The  property  that  is  exploited  is  that  there  are  no  long  carry's  in 

addition.  Thus,  if  we  want  to  add  two  Avizienis  coded  integers  a  =  a  a 

k  k-1 

ai  anc*  b  =  ^k^k-1*  *  *^l  *  we  can  tw0  steps  using  the  following 


3,3.1  Algorithm  (Avizienis) 

(1)  Find  the  carry  c  and  intermediate  sum  r  such  that 


V*!  '  3ci+rl 


where  €  (-2, -1,0, 1,2]  and  c^ ,  r^  €  (-1,0,1) 


(2)  Compute  the  sum  s  according  to 


ai  ■ 


Let  us  estimate  the  length  of  the  formula  representing  any  ternary  place 
in  the  Avizienis  representation  of  for  X  =  (Xj,...,x  ]. 

Again  let  X  =  Y  U  Z  and  Y  fl  Z  =  <t>.  r^  (X)  and  c^(X)  can  be  represented 
as 

Rt00  =  P(R1(Y),R1(z),c1_1(Y),ci_1(z)) 

and 

Ct(X)  =  X(R1(Y),R1(Z),C1_1(Y),C1_1(Z)) 

If  X  is  a  singleton,  and  c^  are  0  if  i  >  1,  or  1  and  0  respectively  if 
i  =  1.  P  and  X  are  certain  operators  (--1,0,1)^  -»  (-1,0,1)  which  can  be 
obtained  from  the  definition  of  Algorithm  3.3.1.  Strictly  speaking,  the 
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domain  used  here  is  not  permitted  in  our  definition  of  finite  functions; 
however,  the  difference  is  merely  one  of  coding.  Thus, 


L(Rt(X))  =  L(R1(Y)+L(Ri(Z)+L(C1-1(Y))+L(C  (Z» 

and 

UC^X))  =  L(Ri(Y))+L(Ri(Z))+L(Ci_1(Y))+L(Ci_1(Z)) 

If  we  use  these  relations  recursively  and  always  make  Y  and  Z  as  equal  as 
possible,  we  obtain 

0(L(Ri)),  0(L(Ct))  *  n2 

for  1  5  i  5  riog3n1.  If  D  =  {0,1]  ,  we  need  two  bits  to  encode  r^  and  c^. 

Therefore,  using  certain  operators  p * ,  p",  X' ,  and  X"  to  encode  P  and  X, 

we  can  encode  R^(X)  and  C^(X)  and  combine  them  into  a  { 0, 1] -formula 
t  h 

representing  the  i  ternary  place  of  the  Avizienis  representation  of  Nj. 
We  have 


0(L(Ai))  *  n‘ 


(3.3.5) 


Let  there  be  given  a  positive  Avizienis  coded  number  a  =  a  a  ,...a,. 

P  P-1  i 

We  desire  to  convert  it  into  its  binary  equivalent  b  =  b  b  _j...bj.  Let 


U  5  (a  ,...,a^].  Then  we  define  b^  (U)  =  1  if  and  only  if  the  I*"*1  bit  of 

E  a. 3*  ^  is  1.  Note  that  even  if  a  is  positive,  b  (U)  may  be  negative 
a.  €U  1  1 

l 

for  some  i  and  U.  This  is  further  discussed  below.  For  the  moment  we  assume 


that  b^(U)  is  always  positive.  We  can  then  again  compute  b^(a  ...a^)  by  a 


.th 


recursive  method.  Let  U=VUw,  V  ^1  W  =  1  Then  b^(U)  is  the  i  bit  of 


the  sum  of  b  (V)...b1(V)  and  b  (W)...b,(W). 

q  1  q  1 
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b^(U)  is  represented  by  the  formula  Bi 

Bt(U)  =  P(Bi(V),Bi(W),Gi.1(U)) 
where  represents  the  carry  from  the  i  place; 

Gi°°  =  Y(Bi(V),Bi(W),Gi_1(U» 

G0  represents  the  constant  0  and  {3  and  y  are  certain  Boolean  operators.  Then 
we  obtain 

i 

L(B  (U))  «  I  L(B  (V))+L(B.(W)) 
j=l  J  J 

If  a  is  the  Avizienis  representation  of  a  number  £  n  then  p  =  Tlog^nl  and 
q  =  riog2nl.  Thus,  we  obtain  the  following  bound  for  L(B,,(a)) 

log  log  n 

0(L(Bi(a)))  *  (2q)  3  « 

log  log  n 
(21og2n)  3 


(l+log2log2n) log2log3n 
log2n 

w  n 


(3.3.6) 


Note  that  (3.3.6)  means  that  0(L(Bi(a)))  £  n  for  li  i  i  Tiog^l  where 

€  -*  0  as  n  -»  “. 

It  has  already  been  remarked  that  b(U)  need  not  be  positive.  Thus 
b(U)  must  be  treated  as  a  signed  number.  If  we  use  the  l's  complement 
representation  and  the  en-around  carry  technique  (see,  e.g.,  [Gr59]),  addition 
can  be  performed  as  follows.  Let  b  (U,gQ)=G(V)+b  (W)+g^  where  is  either  0 
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or  1  and  let  denote  the  carry  from  the  highest  position  of  b(U,0).  Then 
b(U)  =  b(U,0)  if  g^  =  0  and  b(U,l)  If  g^  -  1.  This  means  that  in  (3.3.4) 
we  obtain  (4.q)  ^  for  some  constant  &  instead  of  (Zq)6*^  where  exp  has  the 
value  given  above. 

If  a^  for  1  ^  j  £  Tlog^nl.in  for  1  ^  i  £  ric^nl  is  replaced  with 
Aj,  we  obtain  formulas  F^(X)  representing  in  binary  form.  Combining 
(3.3.5)  and  (3.3.6)  we  obtain 

0(L(F1(X)>  )))  £  n3+£(N) 
where  e(n)  -»  0  as  n  -*  °°. 

To  obtain  the  desired  representation  of  an  arbitrary  Boolean  synmetrlc 
function,  we  proceed  as  follows:  Consider  the  formula  defined  inductively 


*10  V  X11 


Si+1  “  Xi0  A  Si-1  V  xil  A  Si-1 


(3.3.7) 


Take  n1  and  replace  x^q  and  x^  by  and  F^  respectively.  It  is  easily 


seen 


that  S  n-|  with  this  replacement  is  identically  1  (this  can  be  proved, 
e.g. ,  by  induction;  for  it  is  trivially  true,  and  the  general  statement 
follows  from  (3.3.7)). 

Let  there  be  given  an  arbitrary  symnetric  function  g.  It  Is  defined  by 
a  subset  M  C  (0,l,...,n]  of  possible  values  of  N^.  Each  branch  of  length 
riogjnl  in  T(S|-^o^  ^-| )  corresponds  to  one  value  of  (given  by  the  binary 


number  j  (  flog^l ) ,. . . ,  j  (1)  where  xflog  n1 ,  j  (  Tiog-nT) . xl,j(l) 


define  the 


branch  in  question).  If  we  remove  branches  of  T(Sp^o^  n"|)  corresponding  to 
M,  thus  obtaining  the  formula  S',  and  perform  the  substitution  defined  pre~ 


viously  to  obtain  the  formula  p,  we  obtain  a  representation  for  g. 
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We  have  0(L(S^))  £  n  and  thus  0(L(P))  £  n4+S(n)  where  e  -*  0  as  n  -»  00  . 
Thus,  if  a  basis  $  is  given  that  contains  all  operators  used  to  obtain  P, 
then  L(g,$)  £  n4*®0^* 

In  Lu70  Lupanov  announced  a  result  of  Khrapchenko  to  the  effect  that  an 

4  93 

arbitrary  symmetric  function  is  of  length  s  n  *  Since  the  assumption  were 
not  made  explicit,  and  the  result  itself  is  unavailable  as  of  this  writing, 
no  exact  comparison  can  be  made  with  the  estimate  above. 
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where  has  the  I^-property  (only  arrows  labeled  with 

0  and  1  are  drawn).  P  denotes  the  set  of  nodes  <¥>  (x  x 

i  m  m-1 

xo0»a.)  where  m  is  as  described  in  the  text  and  x  ,  ...  ,  x„ 
z  1  m  2 

may  assume  arbitrary  values  in  (0,  1). 

Fig.  3.1 


00000000 
00001111 
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00001001 
01101011 
0  11110  11 
00000000 
00000000 


A  connected  pattern  of  l's 
Fig.  3.2 
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Fig.  3.3a 
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Fig.  3.4a 
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Fig.  3.4b 
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CHAPTER  FOUR 

CYCLIC  PERCEPTRONS 

The  perceptron  has  already  been  discussed  in  1.6.  In  the  beginning 
of  this  chapter,  we  will  first  expand  on  that  discussion  in  order  to  further 
motivate  the  study  of  cyclic  perceptrons. 

The  classical  perceptron  (for  references  on  the  subject  see  [Mi69] 
became  the  subject  of  extensive  research  centered  around  concepts  such  as 
pattern  recognition,  learning,  adaptive  behavior,  etc.  A  whole  myth  had 
been  created  around  it  --  about  its  capabilities  and  its  potential  for  use. 

The  thing  that  attracted  people  most  were  its  ability  to  learn  from  experience 
and  its  simplicity  —  it  combines  many  small  decisions,  the  values  of  the 

functions  into  a  final  decision  by  considering  their  weighted  sum. 

Minsky  and  Paper  deflated  this  myth  by  showing  that  such  a  scheme  has  its 

inherent  drawbacks.  In  particular,  it  cannot  compute  predicates  such  as 

connectivity. 

The  most  general  intuitive  basis  for  the  result  that  the  connectivity 
preciate  cannot  be  represented  by  a  perceptron  is  the  following:  First  of  all 
the  reasoning  makes  sense  only  if  the  complexity  of  the  functions  cd.  is  limited 
in  some  way;  if  not,  we  can  choose  to  be  the  function  that  we  desire  to 
represent  and  then  it  can  be  represented  by  a  perceptron  trivially.  Minsky  and 
Papert  use  the  order  and  diameter  restrictions  (see  [Mi69]).  The  former  is  also 
used  by  us. 

Suppose  we  want  to  represent  connectivity.  Then,  if  the  rr^'s  are  bounded 
in  complexity  (so  the  reasoning  goes),  the  weighted  sum  is  too  simple  a  function 
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to  be  able  to  integrate  all  the  information  that  is  required  in  computing 
connectivity. 

We  set  out  to  apply  the  same  basic  reasoning  to  models  where  the  inte¬ 
grating  function  is  constructed  out  of  finite  operators.  In  particular, 
we  choose  addition  in  a  finite  field  because  of  the  unique  representation 
property  for  functions  in  such  a  field  (see  4.5)  which  makes  proofs  rather 
simple,  and  because  of  the  purely  formal  resenblance  to  perceptrons. 

One  particularly  interesting  aspect  of  using  addition  in  a  finite  field  as 
the  integrating  function  is  that  one  proof  of  the  inability  of  perceptrons  to 
compute  connectivity  is  based  on  the  reduction  of  connectivity  to  addition 
mod  2.  However,  this  function  is  precisely  the  simplest  one  possible  in  GF(2). 
This  underscores  the  need  to  make  different  reductions  for  different  models 
of  computation  that  are  presumed  to  be  incapable  of  computing  connectivity. 

In  this  chapter  we  shall  limit  ourselves  to  Boolean  functions. 

We  introduce  cyclic  perceptrons  formally: 

4.1  Definition 

GF (p  )  is  the  finite  field  consisting  of  pK  elements.  $  (the  basis) 

is  an  infinite  set  of  Boolean  functions  (0,1]U  -»  (0,1]  such  that  each  ©  €  $ 

(w  is  the  first  infinite  ordinal)  depends  on  a  finite  number  of  arguments. 

Elements  of  $  are  assumed  to  be  ordered  (in  an  arbitrary  way).  Then  a  (p  k)- 

perceptron  (over  ?)  is  a  pair  P  =  (a,Y),  where  a  is  an  w-vector  such  that  the 
. th  .  kv 

J.  component  a^  GF(p  )  and  a^O  for  only  finitely  many  values  of  i; 

Y  S  GF (pk) . 
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Given  a  function  f:  (0,1]U  -»  (0,1],  we  will  denote  the  set  of  arguments 
on  which  it  depends  by  S(f). 

Let  P  -  (a,Y)  be  a  (p,k)-perceptron.  Then  P  will  represent  the  predicate 
(Boolean  function) 

CD 

f  ■  r  £  a  cp  €  Y]  (4.1) 

i=0 

where  the  value  of  cp.  6  (0,1]  C  GF(pk).  Obviously,  S(f)  C  U  S^) 

i6(j:  0*0] 

We  will  indicate  the  function  represented  by  a  (p ,k)-perceptron  P  as  in 
(4.1),  or  simply  [P], 

Let  us  recall  a  concept  from  [Mi69].  Given  a  (p ,k)-perceptron  P  =  (a,Y) 

over  a  certain  basis  $,  its  order  (ord(P))  is  max  (S^)  . 

i^(j:  aj^0] 

We  can  also  introduce  the  order  of  a  function. 

4.2  Definition 

The  (p,k)-order  of  a  Boolean  function  f  over  a  given  basis  $  ( (p ,k)-ord$(f 
is  the  smallest  l  such  that  there  exists  a  (p  ,k)-pereeptron  of  order  l  repre¬ 
senting  f.  If  no  such  perceptron  exists,  the  (p,k)-ordet  of  f  is  defined  to 
be  °°. 


Let  0  be  the  set  of  all  Boolean  functions  with  finite  support.  Note  then 
that  for  an  arbitrary  Boolean  function  f,  (p ,k)-ord^(f )  is  finite  and  £  S(f), 
for  all  primes  p  and  arbitrary  k.  Also  note  that  for  an  arbitrary  basis  $ 

(p,k)-ordn(f)  £  (p,k)-ord?(f) 


(4.2) 
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We  show  now,  as  is  done  in  [Mi69],  that  we  can  choose  for  the  basis  a 

more  restricted  set.  Let  the  set  of  arguments  of  the  basis  functions  be  S  = 

(xlfx2,...];  then  we  define  the  set  of  masks  M  =  {  A  x. :  S  is  a  finite  sub- 

i€S  1 

set  of  IM  } .  A  convenient  way  of  ordering  M  is  to  assign  to  cp€m  the  binary 

number  bjb^1...b1  where  t>k  =-  1  if  and  only  if  x]{  appears  in  the  conjunction 
defining  cp. 


4.3  Proposition 

Any  Boolean  function  f  can  be  represented  by  a  (p.k)-perceptron  over  M 
for  any  prime  p,  and  arbitrary  k. 

The  proof  is  the  same  as  that  of  Theorem  1.5.1  in  [Mi69],  i.e. ,  we  util¬ 
ize  the  following  correspondence  between  Boolean  operations  and  operations  in 
GF(p  )  if  the  variables  assume  only  the  values  0  and  1: 


X1  A  X2  ~  X1  *  x2’  X1  V  x2  ~  X1  +  x2  ”  xi  *  x2*  x  ~  1  -  x 

If  f  is  a  function  of  n  arguments,  then  from  its  disjunctive  normal  form,  by 
using  this  correspondence  and  by  multiplying  out  afterwards,  we  obtain  the 
following  representations  for  f: 


2n-l 

I 

i=0 


Vi 


il 


1 


where  is  the  jth  bit  of  the  binary  representation  of  i  and  €  GF(pk). 
Note  that  the  mask  (see  the  ordering  above)  is  represented  by  the  monomial 
with  exponents  corresponding  to  the  binary  representation  of  i. 
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Theorem  1.5.3  of  [Mi.69]  also  holds  in  our  case.  We  state  it  as 
4.4  Proposition 

The  following  holds  for  an  arbitrary  Boolean  function  f,  an  arbitrary 
basis  $,  and  an  arbitrary  integer  k  and  prime  p: 

(p,k)-ord>1(f)  £  (p,k)-ords(f) 


Proof 


The  same  as  in  [Mi69]. 


Note  that  if  we  take  fi  for  the  basis  in  Proposition  4.4,  and  combine  it 
with  (4.2),  we  obtain  that  (p ,k)-perceptrons  over  M  achieve  minimal  order. 

We  state  without  proof  the  following  well-known 

4  5  Lenina 

Every  function  GF(pk)n  -♦  GF(pk)  can  be  uniquely  represented  as  a  polynomial 
in  n  variables  over  GF(pk)  that  is  at  most  of  degree  pk-l  in  each  variable. 

(see,  e.g.,  [La67].) 

It  has  already  been  noted  that  we  will  be  interested  in  whether  a  function 
can  be  represented  by  a  (p ,k)-perceptron  with  a  limitation  on  its  order.  For 
this  we  need  the  following 

4,6  Definition 

A  sequence  of  Boolean  functions  f  ,f2>...  of  1,2,...  arguments  is  of 
(p,k)-order  (over  a  given  basis  $)  if  there  exists  a  finite  r  such 


Bounded  would  be  a  better  word,  but  we  conform  to  the  terminology  of  [Mi69] 
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that  for  all  i  (p,k)-ord$(fi>  £  r. 

Let  there  be  given  a  (p,k)-perceptron  (a,Y).  If  Y  =  ^ . .  ] ,  then, 

recalling  that  GF(p  )  is  a  vector  space  of  dimension  k  over  GF(p),  and  desig¬ 
nating  the  j  component  of  a.  by  a  (similarly  for  y,  €  Y),  we  have 

1  h 


[  E  a.  co.  €  Y] 
i=0  1  1 


m  k 
V  A 
h=l  j=l 


r  Jo  ^  •»  =  V 


(4.3) 


We  can  restrict  the  diversity  of  perceptrons  we  are  dealing  with  by  noting 
4.7  Proposition 

Let  $  be  a  basis  closed  under  conjunction  (i.e. ,  co,  i|f  6  $  =>  cp  A  €  $). 
If  a  Boolean  function  f  is  of  finite  (p.k)-order  over  ?,  then  it  is  of  finite 
(p,l)-order  (but  the  order  may  change). 


Proof 

We  have  f  =  f(a,Y)]  where  (a,Y)  is  a  (p,k)-perceptron.  Suppose  |y|  =m 
and  the  (p.k)-order  of  f  is  i.  Prom(!f.3)  we  have 


m 

f  =  V 
h=l 


k 

A 

j=l 


f  S  airwi  = 

i=0  iJ  1  hJ 


(4.4) 


where  a^,  yhJ  6  GF(p). 

By  Lenina  4.5,  we  know  that  for  all  a  €  GF(p)  there  always  exists  a  poly¬ 
nomial  Pfl(x)  over  GF(p)  of  degree  p-1  which  takes  on  the  value  of  1  if  x  =  a 
and  is  0  otherwise  (the  degree  follows  from  the  number  of  zeros  of  the  poly¬ 
nomial).  Thus  substituting  the  Boolean  operations  with  the  field  operations 
introduced  in  the  proof  of  Proposition  4.3,  we  obtain  from  (4.4) 
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f 


k  00 

q(  n  p  <  s 

j=l  yij  1=0 


y 


k 

n  p 

j=l  ymj 


00 


(  s 

1=0 


V’i” 


(4.5) 


where  Q(x^, . . . ,xn)  is  the  polynomial  (of  degree  m)  that  represents  the  Boolean 
m  f 

function  V  x.  .  Each  P  Is  of  degree  p-1.  Hence  f  can  be  expressed  as 
h=l  n  yij 

a  polynomial  In  the  cp± ' s  of  degree  £  m*  (p-1).  Obviously,  cpjj  for  j  >  1  can  be 
replaced  by  since  it  assumes  only  the  values  0  and  1.  Also,  cp*  \|r  represents 
the  function  cpAi|r  and  |s(«p  A  $)|  £  | S (cp>  |  +  |S(^i)|);  thus,  if  the  basis  is  closed 
under  conjunction  (as,  e.g. ,  0  or  M),  (4.5)  describes  a  (p , l)-perceptron  for  f 
of  order  ^m*(p-l)*Ji.  □ 


Remark 

Incidentally,  this  proof  also  shows  th»t  we  can  assume  the  cardinality 
of  Y  to  be  1. 

Since  we  shall  subsequently  be  concerned  only  in  whether  the  order  of  cer¬ 
tain  functions  is  finite  or  not,  we  will  be  able  to  limit  ourselves  to  (p,l)- 
perceptrons.  For  convenience,  we  will  write  simply  "p-perceptrons".  Also,  we 
will  be  only  concerned  in  whether  there  exists  a  basis  over  which  a  function  is 
of  finite  order.  This  is  equivalent  to  whether  a  function  is  of  finite  order 
over  M. 


t 

Q(xlf. . .  ,xm)  is  obtained  by  using  V  Y2  ~  ”  yl  *  y2  recursiv‘i-1y  ; 

i .  e. ,  Q(x^ ,  •  •  •  ,  x^)  Q(x^ ,  •  • .  ,  x^_  ^  x^  +  x^*  Q(^^»*** » If  Qis  a  poly¬ 

nomial  over  GF(2),  then  Q  =  ©  II  y  where  the  sum  ranges  over  all  nonempty 

y€s 

subsets  S  s  ]. 

1  1  ’  m 
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We  first  turn  our  attention  to  the  case  when  p  =  2.  Instead  of  "2-percep- 
tron",  we  will  say  ’’Boolean  perceptron”. 

From  Lenina  4.5  we  conclude  that  every  Boolean  function  can  be  uniquely 
represented  as  a  polynomial  over  GF(2)  that  is  at  most  of  degree  one  in  each 
variable  (a  Boolean  polynomial). 

Noting  that  the  terms  of  a  Boolean  polynomial  represent  marks,  we  conclude 
that  every  Boolean  function  f  has  a  unique  representation  as  a  Boolean  perceptron 
over  M.  Furthermore,  by  Proposition  4.4,  this  representation  is  a  minimal  order 
representation  for  f.  Note  then  that  2-ord^(f)  corresponds  to  the  degree  of 
the  Boolean  polynomial  for  f. 

This  unique  representation  property  allows  us  to  establish  the  minimal 

order  of  certain  interesting  predicates  very  easily.  As  in  3.2,  we  are  again 

2 

interested  only  in  functions  (0,l]n  -4  (0,1)  that  are  interpreted  as  functions 

of  nxn  patterns  of  0's  and  l's.  In  particular,  we  are  interested  in  the  Boolean 
2 

function  of  n  variables  c  (introduced  in  3.2)  and  e  ,  (the  Euler  number  of 

n  n,k 

a  pattern  of  l's  on  a  square  array  of  0's  and  l's  is  equal  to  k).  It  is  well 
known  (see,  for  example  [Mi69])  that  the  Euler  number  of  a  planar  figure  is 
the  difference  between  the  number  of  its  components  and  the  number  of  its  holes. 

If  we  use  the  notion  of  connectivity  introduced  in  3.2,  then  the  Euler  number 
of  the  pattern  in  Fig.  4.1  is  1. 

4.8  Theorem 

The  connectivity  predicate  is  not  of  finite  2-order  over  M  (hence,  over 


any  basis). 
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Proof 

We  use  the  One-in-a-box  construction  introduced  in  [Mi69].  Before  pro¬ 
ceeding,  however,  we  must  define  certain  auxiliary  predicates,  n,  the  size 
of  the  pattern  is  assume  odd  (henceforth,  we  will  suppress  the  subscript  n 
in  the  notation  for  functions).  The  variables  representing  positions  in  the 
square  array  will,  as  usual,  be  denoted  by  x£j  for  1  £  i,  j  *  n.  Then  we  define 


and 


r  =  (X11  A  X12  A  •••  A  xln)  A 

(X31  A  X32  A  “•  A  x3n^  A  A 

<Xnl  AXn2  A-  Axnn> 


S  *»  (X21  V  x22  V  ...  V  X„_)  A 


(X41  V  *42 


2n> 

V  •  •  •  V  x4n  ^  A  • • •  A 


<*  i  1  V  *  ,  9  V  ...  V  x  ); 
n-i,i  n-1,2  n-l,n  ’ 


i.e.,  r  is  1  only  on  patterns  with  odd  rows  consisting  exclusively  of  l's, 
and  s  is  1  only  on  patterns  where  each  even  row  has  at  least  one  1  (the  One- 
in-a-box  predicate).  Then, 

r  A  c  -  r  A  s  (4.6) 

(c  is  the  connectivity  predicate). 

Now,  for  arbitrary  functions f,  g,  h,if  h  =  f  A  g,  then  2-ord%  (h)  £  2-ord 
(f)+2-ordM(g);  i.e, , 


2-ordM(g)  £  2-ordM(h)-2-ordM(f) 


(4.7) 
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Replacing  h  by  r  A  s,  f  by  r,  and  g  by  c  we  obtain 

2"ordM(c)  *  2-ordM^r  A  s)“2“°rd^(r)  (4.8) 

We  have  2-ordH<r)  =  -n;  2-ord^h)  -  Ei  •„  (recall  the  Boolean 

m 

polynomial  for  ^  Xi  described  in  the  footnote  on  p.  80  )  2-ord^r  A  s )  = 
n(n-l)  (because  ;the  Boolean  polynomial  representations  of  r  and  s  have  no 
variables  in  common).  Using  this  we  obtain  from  (4.8)  2-ord^c)  a  . 

l.e. ,  the  2-order  over  M  of  the  connectivity  predicate  is  not  finite.  □ 

We  next  establish 

4. 9  Theorem 

The  predicate  "the  Euler  number  of  a  pattern  equals  k"  is  not  of  finite 
2-order. 

Proof 

W«  again  consider  the  case  when  M  is  the  basis.  The  general  case  follows 
from  Proposition  4.4.  n  is  the  size  of  the  pattern.  We  need  to  consider  a 
subset  T  C  S  =  (xlj:  i+j  even]  (note  that  all  points  of  S  are  disconnected 

from  each  other,  in  the  sense  we  use  this  word).  |t|  =  t  will  be  determined 
subsequently. 

We  define  the  following  predicates 

Pr  =  1  if  and  only  if  all  points  of  T  are  0;  i.e., 

pr  =  H  (1  ®x) 
x?T 

q  =  1  if  and  only  if  k  points  of  T  are  1;  i.e., 
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q  =  ©  n  x  •  n  (l®y) 
x€U  y€T-U 

where  the  sum  ranges  over  all  possible  subsets  U  c  T  with  |u|  =  k.  When  the 

expression  for  q  is  multiplied  out  each  term  produces  exactly  one  term  of  the 

form  II  x  and  thus  the  above  Boolean  polynomial  is  of  degree  t  if  and  only 
xcT  3 

if  the  number  of  terms  in  the  sum  is  odd.  The  number  of  terms  is  (C).  But 

2-1  4  +  k 

(  )  is  odd  for  all  0  s  k  s  2  «1  and  all  J J.  Thus  if  t  =  2*-l,  0  £  k  £  t 

k 

then  2-ord (q)  =  t.  Also,  2-ord(pr)  =  |t |  =  n2-t. 

Recalling  once  again  the  e^  is  the  difference  between  the  number  of  compo¬ 
nents  of  a  figure  and  the  number  of  holes,  wa  have  the  relationship 

pr  A  =  pr  A  q 


Again  using  (4.7)  with  g  *  e^*  h  *  pr  A  q,  f  ■  pr  we  obtain 

2-ordM(ek)  *  n2  -  |t!  -  2^-1 

No  matter  how  large  we  choose  !>,  we  can  find  an  n  such  that  we  can  obtain 


a  set  T  with  T)  =  2  -1.  Thus,  the  Euler  predicate  is  not  of  finite  order 


over  M. 


Theorems  4,8  and  4.9  can  be  extended  to  p-perceptrons  for  arbitrary 
p.  The  generalization  will  only  be  indicated  for  Theorem  4.8. 


t  “  o 

Proof:  First  show  that  (  )  is  even  for  all  J i  and  all  MO,  2m.  This  is  done 

by  induction.  Now  observe  that  due  to  (")  =  C"’1)  +  (”"*),  and  the  fact  that 
l  i  R  k  .  k~  1 

2-1  2-1  2^ 

(  ^  )  is  odd,  (  9  )  is  also  odd  (for  otherwise  )  would  not  be  even).  We 


can  continue  this  way  and  establish  the  claim. 
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The  obvious  difficulty  is  that  Boolean  functions  do  not  have  a  unique 
representation  as  polynomials  over  GF(p)  for  p  >  2.  Specifically,  in  the 
case  of  Boolean  perceptrons  over  M,  we  were  able  to  reduce  the  problem  of 
the  order  of  connectivity  to  the  order  of  r  and  s  (see  above).  The  orders 
of  these  predicates  (equal  to  the  degrees  of  the  corresponding  Boolean  poly¬ 
nomials)  were  easily  computed  due  to  Lenina  4.5. 

Suppose  c  is  of  finite  p-order  over  M  (we  have  already  remarked  that  this 
brings  no  loss  of  generality)  for  some  p  >  2.  Due  to  Proposition  4.7  we  can 
assume  that  we  have  an  expression  of  the  form  of  (4.5)  for  c^.  When  multiplied 
out,  we  obtain 

00 

c  =■  E  a.m  mod  p  (4.9) 

n  i-0  1  1 

where  m^  is  the  monomial  representing  the  i  mask,  m^  is  of  degree  one  in 
each  variable,  and  the  values  of  the  variables  are  restricted  to  (0,1).  Since 
the  perceptron  from  which  we  obtained  (4.9)  is  finite  order,  we  can  assume  that 
the  degree  of  (4.9)  is  £  l. 

We  can  now  extend  cr  to  the  domain  GF)p)  (i.e.,  to  square  patterns  A  = 

(b^j),  1  S  i,  j  S  n,  and  b^  €  GF(p))  by  defining  c^(A)  *  1  if  and  only  if 

c  (f(A))  =  1  where  f(b..])  =  (c.  .}  such  that  c. .  =  1  if  b.  =  1,  otherwise 
n'  ij  ij  ij  ij 

Cij  *  ^aVR  ^rtnn  (4.9) 

00 

c'  =  T.  a.  .m'  mod  p  (4.10) 

n  i=0  1  1 


where  ra^  is  obtained  from  by  replacing  each  variable  x^  by  P^x^)  such  that 

P^(Xj)  =  1,  otherwise  P^(Xj)  =  0  (see  the  proof  of  Proposition  4.7  how  to  ob¬ 
tain  P^(Xj)).  Now  we  have  a  total  function  c^  over  GF(p),  and  thus  its 
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polynomial  representation  obtained  by  multiplying  out  (4.10)  must  be  unique. 
Since  the  degree  of  (4.9)  is  £  i, 

the  degree  of  the  polynomial  (4.10)  s  (p-l)X  (4.11) 

Another  estimate  of  the  degree  of  the  polynomial  for  c'  is  obtained  using 

the  predicates  r'  and  s'  obtained  from  r  and  s  of  Theorem  4.8  similarly  as 

cn  was  obtained  from  cr.  The  polynomial  Pr ,  representing  r',  is  obtained  from 

the  polynomial  representing  r  by  substituting  each  variable  x  with  P^x). 

Similarly,  for  the  polynomial  representation  P  ,  of  s'.  The  degrees  of  P 

s  r 

and  Pgl  are  then  found  to  be  ^  ^  n-  (p-1)  and  *  ^  n(p-l)  respectively 

(2*ordM(r)  and  2-ord^s)  multiplied  by  deg^)).  Since  the  analogs  of  (4.6) 

aad  (4.8)  again  hold,  we  obtain  that  deg(Pc,)  is  not  bounded,  contradicting 

(4*H)»and  thus  also  the  finite  order  of  c. 

Note  that  the  obvious  generalization,  i.e.f  if  a  function  is  of  finite 

2-order,  then  it  is  also  of  finite  p-order  (over  M),  is  not  true:  Consider 

n 

the  Boolean  function  P  x^  We  will  investigate  the  degree  of  the  polynomial 

i=l 

representation  of  P'  (obtained  in  the  same  way  from  ®aa  c1  was  from  c  above). 
If  a  p-perceptron  over  M  of  finite  order  exists  for  P,  then  we  obtain  a  poly¬ 
nomial  representation  of  bounded  degree  for  P'  similarly  as  (4.10)  was 
obtained  from  (4.9).  On  the  other  hand  we  have  the  following  representation 
for  ® 


I  II  x  II  (1-x  )  mod  p  (4.12) 

i€S  i?S  3 

where  S  ranges  over  all  subsets  of  (l,...,n]  of  even  size.  When  multiplied 

n 

out,  each  term  produces  exactly  one  monomial  of  the  form  n  x . . 

i=l  1 


The  number 
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of  such  monomials  appearing  In  the  developed  form  of  (4.12)  Is 


(use  the  Identity  (^)  *  (n^)  +  (J_j^)  )•  Since  this  number  Is  not  0 

mod  p,  (4.12)  yields  a  p-perceptron  over  M  of  order  n  for  <®.  If  x^  Is 
replaced  by  P^(x^)  we  obtain  a  (unique)  polynomial  representation  for 
C&'  of  degree  n*(p-l),  contradicting  the  existence  of  a  finite  order 
p-perceptron  over  M  for  ffi. 
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CHAPTER  FIVE 

PATTERN  COUNTING  MACHINES 

In  this  chapter  we  shall  permit  ourselves  a  certain  degree  of  informality. 

We  are  again  concerned  with  the  power  of  machines  that  combine  a  large 
number  of  "local"  computations  through  an  integrating  function.  Only  this 
time  we  shall  not  be  limited  to  functions  that  can  be  represented  as  a  combin¬ 
ation  of  finite  operators. 

This  class  of  machines  again  operates  on  square  patterns  of  0's  and  l's. 
The  operation  is  divided  into  two  phases:  In  Phase  I  the  pattern  is  scanned 
with  a  square  "window"  of  a  certain  size.  Each  time  a  nonzero  pattern  appears 
in  the  window,  we  take  note  of  it  (there  is  a  finite  number  of  nonzero  patterns 
since  the  window  is  of  finite  size).  At  the  end  of  the  scan  we  have  a  count 
of  the  various  patterns, and  we  are  then  allowed  to  utilize  this  data  in  Phase 
II  which  consists  of  computing  the  value  of  a  partial  recursive  function  for 
this  data.  The  formalization  of  this  model  is  obvious  and  we  omit  it. 

What  can  su<-’  a  machine  do?  Clearly  the  computation  of  this  machine  is 
divided  into  a  local  phase  and  a  global  phase,  so  that  it  fits  into  the  broad 
class  of  problems  considered  in  [Mi69]  and  Chapter  Four. 

Note  that  the  boundedness  of  the  window  size  is  essential.  If  we  insis¬ 
ted  only  that  the  window  contain  a  given  number  of  points,  but  otherwise 
allowed  it  to  be  of  any  shape  with  arbitrary  distances  between  its  points, 
then  Phase  II  could  reconstruct  the  whole  figure  as  was  observed  already  in 
C  Mi  69] . 

We  again  inquire  whether  these  machines  can  recognize  the  familiar 
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topological  predicates  connectivity  and  Euler  number. 

5. 1  Theorem 

Pattern  Counting  Machines  (PCM)  cannot  recognise  the  connectivity  pred¬ 
icate. 

Proof 

We  need  only  exhibit  two  patterns,  one  connected  and  the  other  discon¬ 
nected,  with  the  uame  pattern  spectrum.  In  this  case,  no  algorithm  of  Phase 
II  could  establish  the  difference  between  them. 

Two  such  patterns  are  given  in  Fig.  5.1. 

Specifically,  these  patterns  are  equivalent  under  windows  of  sise  2x2. 
However,  it  is  easily  seen  that  increasing  the  dimensions  of  the  patterns 
in  Fig.  5. 1  linearly  by  a  factor  of  k  makes  them  equivalent  under  windows 
of  size  up  to  k  +  1.  We  can  arrive  at  this  conclusion  by  setting  up  a  1-1 
map  between  occurrences  of  the  same  pattern  in  the  window  in  the  two  pat¬ 
terns 

5 . 2  Theorem 

PCM' s  can  compute  the  Euler  number. 

Proof 

It  is  shown  in  fMi69]  how  to  compute  the  Euler  number  from  the  spectrum 
of  patterns  of  the  shape 

3  m  0  □ 


D 


91 


Before  proceeding,  we  need  a  notion  of  continuous  deformation.  Pattern 
B  can  be  obtained  from  p t  tern  A  by  continuous  deformation  if  B  arises  from 
A  by  a  sequence  of  additions  or  deletions  of  l's  of  the  following  kind:  Let 
us  fix  attention  on  a  3x3  square  with  the  central  position  in  the  place  of 
the  1  being  added  (deleted).  For  simplicity  assume  that  the  boundary  positions 
are  always  0.  Each  position  of  the  periphery  of  the  square  which  has  a  1  in 
it  is  either  connected  or  disconnected  to  another  1  on  the  periphery  (not 
necessarily  by  a  path  in  the  square).  This  set  of  connections  may  be  described 

by  a  symmetric  8x8  0-1  valued  connection  matrix,  i.e. ,  a  =  1  if  and  only  if 

tH  tH  J 

the  i  and  j  positions  on  the  periphery  have  l's  and  are  connected.  The 

proposed  addition  (deletion)  is  permitted  cnly  if  (1)  the  connection  matrix 

remains  unchanged  as  a  result  of  it,  and  (2)  there  is  a  1  adjacent  to  the 

proposed  addition  (deletion). 

Any  predicate  whose  value  remains  unchanged  if  the  pattern  A  is  replaced 
by  B,  obtained  from  A  by  continuous  deformation,  is  called  a  topological 
Indicate.  We  assert  without  proof  that  connectivity  and  Euler  number  are 
topological  predicates.  The  reader  is  warned,  however,  that  there  is  a  pitfall 
in  proving  this  fact  for  the  Euler  number  predicate.  The  number  of  holes  in 
Fig.  5.2  should  be  one,  not  two  (i.e.,  0's  are  connected  diagonally  in  addition 
to  their  usual  connections).  This  is  discussed  more  fully  in  [My71J.  However, 
if  the  holes  are  sufficiently  large  (so  that  all  the  0's  in  them  are  connected 
in  the  usual  way)  this  difficulty  is  not  encountered. 

5.3  Theorem 

Any  topological  predicate  recognized  by  a  PCM  must  be  a  function  of  the 


Euler  number. 
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Proof 

We  will  have  established  the  theorem  if  we  succeed  in  showing  that, 
given  any  PCM  p  computing  a  topological  predicate,  then  for  two  figures 
Xl  and  X2  with  EULER  (X^  =  EULER (X2 ),  we  also  have  P^)  =  P(X2). 

In  [Mi69]  it  is  shown  that  for  every  figure  X  there  exists  an  "Euler 
canonical"  figure  C(X)  such  that  EULER(X)  =  EULER(C(X));  and  if  for  two 
figures  X^  X2,  EULER(X1)  =  EULER (X2 )  ,  then  C^)  =  C(X2>.  If  the  Euler 
number  of  X  is  n  >  0,  then  C(X)  consists  of  n  components  without  holes.  If 
the  Euler  number  of  X  is  n  i  0,  then  C(X)  consists  of  1  component  with-n+1 
holes . 

We  will  show  that  we  can  deform  any  figure  X  into  C(X)  without  changing 
the  value  of  P. 

The  deformations  available  to  us  are: 

(1)  Continuous  deformation.  If  we  subject  X  to  this  kind  of  deformation, 
then  P(X)  remains  unchanged  because  it  computes  a  topological  predicate. 

(2)  Deformations  that  leave  the  pattern  spectrum  unaltered.  By  defini¬ 
tion  of  PCM's. 

As  a  consequence,  we  have 

(3)  Removal  of  components  inside  holes.  To  accomplish  this  without 
changing  the  value  of  P(X),  we  first  apply  (1)  until  the  window  cannot  scan 
simultaneously  an  interior  component  and  the  wall  of  the  hole  in  which  it 
resides.  Then  it  is  obvious  that  the  pattern  spectrum  will  remain  unchanged 
if  we  remove  the  component  from  inside  the  hole.  After  this  we  can  apply 
(1)  in  the  reverse  direction. 
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If  we  are  given  two  figures  Xj.  and  X£  with  same  pattern  spectra,  then 
we  can  add  equally  shaped  holes  to  the  figures  in  such  a  way  that  the  pattern 
spectra  remain  the  same.  The  holes  have  only  to  be  placed  in  such  a  way  that 
the  window  cannot  scan  any  other  boundary  while  scanning  the  newly  introduced 
hole.  We  can  then  repeat  this  to  add  any  number  of  holes. 

Specifically,  given  two  figures  of  the  shape  of  A  and  B  in  Fig.  5.1,  we 
can  add  holes  in  this  way  and  still  have  the  same  pattern  spectra.  For  example 
C  and  D  in  Fig.  5.3  have  the  same  pattern  spectra  for  a  sufficiently  small 
window  size.  Note  that  given  any  two  components,  one  of  which  has  a  hole, 
we  may  apply  deformation  (1)  to  obtain  a  figure  proportional  in  dimensions 
to  C  and  they  apply  (2)  to  obtain  D.  We  call  this  sequence  of  deformations 
'cancelling  a  hole  and  a  component". 

We  deform  X  into  C(X)  by  cancelling  as  many  holes  and  components  as 
possible.  We  first  apply  (3)  until  no  component  remains  within  a  hole. 

Then  we  may  either  have  a  hole  and  a  component  not  containing  this  hole,  or 
not.  In  the  latter  case,  we  are  done.  In  the  former,  we  select  a  hole  and 
a  component  not  containing  it  and  cancel  them.  After  this  we  are  left  with 
one  less  component  and  hole.  We  repeat  this  until  we  arrive  at  C(X). 

We  can  summarize  the  results  on  the  recognition  of  topological  predicates 
contained  in  [Mi69],  Chapter  Four,  and  in  the  present  section  in  tte  following 


table 
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Recognition 

of 

Predicates 

Cyclic 

Perceptrons 

PCM 

Classical 

Perceptrons 

Connectivity 

NO 

NO 

NO 

Euler 

NO 

YES 

YES 

Other 

Topological 

Predicates 

? 

Functions  of 
Euler  number 
only 

Functions  of 
Euler  number 
only 

Thus,  all  these  results  support  the  conjecture  expressed  in  [Mi69]  that 
no  "local-global"  computer  can  recognize  connectivity. 

It  appears,  however,  that  all  models  are  extremely  sensitive  to  altera¬ 
tions.  We  have  already  mentioned  how  PCM's  can  be  converted  into  universal 
machines  with  the  removal  of  the  restriction  on  the  size  of  the  window. 

A.  R.  Meyer  noticed  that  ordinary  perceptrons  may  be  modified  to  recognize 

00 

any  Boolean  function  with  order  owe.  Instead  of  E  a  cp  2:  0  consider 

a,  i«0  1  1 

E  €  Y  for  some  subset  of  integers  Y.  Now  we  can  choose  the  coeffi- 

i=0  1  1 

cients  a^  in  such  a  way  that  the  sums  of  the  coefficients  in  no  two  subsets 
of  the  set  K  of  all  coefficients  is  the  same,  ie. , 

V(X,Y  =  K)  [  E  &t  =  E  &i  *  X  =  Y] 
ajL  €  X  ajL  £  Y 

We  can  define  the  coefficients  inductively,  i.e. ,  choose  a  to  be  greater 
n-1  n 

than  ^  aj.  This  is  in  the  spirit  of  stratification  (see  [M169]).  Now 
notice  th&t  if  $  is  the  set  of  mjjks  of  order  1,  then  a  Boolean  function  cp 
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is  simply  a  collection  of  subsets  of  K.  Thus,  Y  can  be  chosen  as  the  set 

of  integers  representing  the  sums  of  coefficients  in  individual  subsets 
belonging  to  ©. 
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Two  figures  with  the  same  2x2  pattern  spectra 
Fig.  5.1 
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The  number  of  holes  In  this  pattern  Is  one 
Fig.  5.2 


Cancelling  a  hole  and  a  component 
Fig.  5.3 
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APPENDIX  A 


CERTAIN  PROPERTIES  OF  SHORT  FORMULAS 


The  purpose  of  this  appendix  is  to  modify  certain  results  of  [H068]  in 
the  light  of  our  different  requirements.  Our  goal  is  Lemma  A. 9  which  is  used 
directly  in  the  proof  of  Theorem  2.2.2.  We  prove  it  by  way  of  a  series  of 
intermediate  results,  none  of  which  are  used  elsewhere. 

In  what  follows  we  would  frequently  use  the  phrase  "F  is  a  formula  in 
n  variables  over  i,  and  such  that  no  variable  occurs  more  than  k  times". 

This  will  be  abbreviated  to  "F  is  a  ($,n,  k)-formula".  If  any  of  the  para¬ 
meters  is  not  present,  we  will  replace  it  by  *.  For  example,  "F  is  a  (®,*,k)- 
formula"  and  "F  is  a  ($,n,*)- formula”  mean  "F  is  a  formula  over  5,  and  such 
that  no  variable  appears  more  than  k  times"  and  "F  is  a  formula  in  n  vari¬ 
ables  over  $n  respectively. 


A.  1  Definition 

Let  there  be  given  the  sequence  of  formulas  G  =  (G^(X^,z) . ^p-1 

(Xp_^,z) ,Gp(Xp)).  If  1  €  1  £  P“l,  then  G^  contains  the  distinguished 

variable  z,  occurring  only  once.  X^  for  1  £  i  £  p  is  nonempty  and  is 

either  a  singleton  or  £  I  )  X  ..  Let  F  be  an  arbitrary  formula,  and 

j<i  j 

G  =  G^(X^,G2(X2 . Gp  i (Xp_iGp (Xp)) . . .  )) .  If  F  =  G,  then  G  is  a  nesting 

sequence  of  length  £  for  F.  If,  in  addition,  the  total  number  of  occur¬ 
rences  of  any  variable  (except  z)  in  G  is  £  the  corresponding  number  in  F, 
then  G  is  a  proper  nesting  sequence  for  F. 
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A. 2  Remark 

Let  G  and  G  be  as  described  in  Definition  A.l.  Furthermore,  let  all 

Xi  for  1  £  i  £  p  be  singletons  and  distinct.  Then  G  is  equivalent  to  an 

ep_ ^component.  Also,  suppose  is  arbitrary  and  G£  is  a  formula  over  f. 

Now  replace  all  variables  except  possibly  one  in  Gj  for  1  S  i  s  p-1  by  the 

constant  a.  Let  the  set  of  variables  that  have  not  been  touched  be  Y. 

Y 

equivalent  to  an  e-component  over  $a. 

Let  F  be  an  arbitrary  formula  over  *,  X?S(F),  and  a  f  D;  then  we 
would  like  to  obtain  a  formula  over  &a  with  the  following  properties: 

(1)  G  =  Ffl  (2)  S(G)  »  X,  and  (3)  the  number  of  occurrences  of  any  vari¬ 
able  of  X  in  G  is  £  the  corresponding  number  in  F.  G  can  be  obtained 
by  a  straightforward  replacement  of  operators  in  F  such  that  the  variable 

symbols  that  are  replaced  with  a  in  forming  FX  (and  subformulas  of  F 

a 

where  S(F)  consists  entirely  of  such  variable  symbols)  are  removed,  and 

the  remaining  operators  are  changed  to  preserve  equivalence  with  FX. 

a 

More  precisely,  if  co(F^ , . . .  ,F^)  is  a  subformula  of  F,  then  if  S(F^)  <£  X 

for  all  i,  cp  remains  the  same;  if  S(Fi)  cx  and  S(Fj)  £X  for  jj«i,  then 

cp  is  replaced  with  cp(x^  , . . .  ,x^_ ^  ,F^  ,X^+^ , . . .  ,x^)  where  all  variables  of 

F^  have  been  replaced  with  a  (if  there  are  more  such  indices  i,  we  proceed 

in  the  obvious  way);  and  if  SfF^c'x  for  all  i,  then  cp  is  eliminated.  This 

transformation  will  be  called  normalization  and  G  will  be  denoted  by  norm(FX). 

a 
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A. 3  Lemma 

If  F  is  a  ($ ,n,*)-formula,  then  for  any  p,  q  ^  1  and  a  €  D,  if 
n  *  \  there  exists  a  subset  X  =  S(F)  such  that  either 

(1)  |x!  =  q  and  FX  is  equivalent  to  a  PC  of  the  formulas  F, , . . . ,F 

a  1 7  9  r 

where  r  ^  n^  and  F{  for  1  s  i  s  r  is  a  formula  over  $a  such  that  each  element 
of  X  occurs  in  at  least  two  among  F^...^,  and  the  total  number  of 

occurrences  of  any  x  c  X  in  Flf...,Fr  is  s  the  number  of  occurrences  of  x 
in  F;  or 

(2)  | X j is  arbitrary  and  F^  has  a  proper  nesting  sequence  G  =  (G^,..., 
Gp-r  V  where  Gi  for  1  s  1  s  P  is  a  formula  over 


Proof 

Assume  there  is  no  x  =  S(F)  such  that  FX  is  as  described  in  (1)  of 
the  statement  of  the  lemma. 

We  will  describe  a  (proper)  nesting  sequence  extraction  procedure 
(NSE)  whose  inputs  will  be  a  formula  H  over  $a  and  a  set  of  variables 
Y.  The  output  of  NSE  will  be  two  formulas  H' (Z,z)  and  Hw  over  $a  such 
that  Z  is  either  a  singleton  or  C  Y;  furthermore,  HU  s  H'(Z,H")  for  some 

EL 

U  C  S(H). 


G  will  be  obtained  by  the  repeated  use  of  NSE.  Initially,  the  input 

of  NSE  will  be  F  =  Fq  and  <t.  In  the  first  application  of  NSE,  the  output 

will  be  G1  and  Fj  (F^  is  an  intermediate  formula  whose  significance  will 

be  describe  immediately).  In  general,  the  ith  application  of  NSE  will 

receive  the  input  F^  and  I  1  X  and  yield  as  output  G  and  F..  We  will 

j<  i  J  11 
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Show  that  if  n  *  Tlg(p,q),  we  can  apply  NSE  p-1  times  and  end  up  with  F^_2 

from  which  G  is  obtained  as  will  be  described  below. 

P 

Description  of  NSE.  The  input  to  NSE  is  as  describe  above.  Then 
we  can  distinquish  two  cases: 

Case  I.  L(H)  =  1.  In  this  case  we  cannot  apply  NSE,  and  the  output 
is  undefined. 

Case  II.  L(H)  >  1.  In  this  case  we  can  assume  that  H  has  no  unary 
operators;  for  suppose  there  exists  a  subformula  J  of  H  such  that  J  =  cd ( ^ ( 
J^,...,Jr))  where  for  1  s  i  S  r  is  either  a  variable  symbol  or  another 

subformula  of  H.  In  this  case  cp*  =  p  G  $  and  we  can  replace  J  by  the 

equivalent  formula  p  (J^ , . . . ,  Jf) .  Similarly,  if  J  *=  CD  (J^ ,. . .  >  ) . 

we  can  eliminate  tjf  because  cp , . . .  ,  'l'  (x^)  , . . .  .x^)  *  P  (x^  , . .  c  ,x^ , . . .  ,x^) 
f  $a  (thus,  if  a  unary  operator  of  H  corresponds  to  an  internal  node  of 
T(H),  we  can  eliminate  it  by  either  of  these  two  means;  on  the  other  hand, 
if  a  unary  operator  of  H  corresponds  either  to  the  root  node  or  to  a  node 
next  to  a  terminal  node  of  T(H),  then  we  can  use  only  one  of  the  two  methods 
described).  Now  choose  i'  such  that  S(H  is  maximal  among  S(H  )  for 
1  s  i  s  r.  Since  support  is  defined  only  for  formulas,  S(H  ^,)  may  be 
undefined  if  all  arguments  of  the  outermost  operator  of  H  are  variable 
symbols.  In  this  case  replace  one  of  them  by  the  identity  operator 
which  is  possible  since  id  £  $a.  Consider  H/H  =  K(Z,z).  Again  two 


cases  can  arise: 
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Case  Ila.  Y  fl  Z  ■  (b.  Choose  any  variable  of  Z,  e.g.  ,  x,  and  let 
H'  *  norm(K  g  ).  z  is  a  distinquished  argument  (hence  x  is  free). 

In  this  case  set  V  =  S(H  J-Z  U(x].  The  significance  of  V  will  be 
seen  immediately. 

Case  lib.  Y  fl  Z  ^  <b,  H'  =  norm(K^'^Z^ )]  z  is  again  a  distinguished 
argument  V  =  S(H  ,)-Z  U  Y. 

•  J- 

In  both  cases  Hw  =  norm((H  ,  )^). 

•  i  d 

Analysis  of  NSE.  Let  ls(H)l  =  m,  and  let  us  estimate  ls(H")l. 

Obviously,  |s(H  ii)|  ^  n-E—  .  In  the  case  that  H  results  from  a  chain 

max 

of  applications  of  NSE  to  a  formula  F,  and  F  does  not  satisfy  (1)  of  the 
statement  of  the  lemma,  then  we  claim  that  in  Cases  Ila  and  b  less  than 


q  variables  are  set  to  a  in  H  . , .  Suppose  this  is  not  true.  Let  the 

set  of  variables  that  is  set  to  a  on  this  occasion  be  W.  Then  W  C  Z-(x) 

(Case  Ila),  or  W  £  Z~Y  (Case  lib).  In  any  case  consider  F^.  This  is 

a 

equivalent  to  ^(H  ^^j)a»...,(H  ^g^)g  where  i(l) , . . .  ,i  (s)  are  the 


indices  corresponding  to  the  subformulas  H 


where  all  variables  have 


not  been  replaced  by  a  (if  in  H  ^  all  variables  have  been  replaced  by  a, 

then  it  is  absorbed  into  cp).  But  then  cp(norm((H  ,  ., .  )^)  , . . .  ,norm((H  ,  ,  .)W)) 

.  i  (,i;  a  .  i  (s ;  a 

satisfies  (1)  of  the  statement  of  the  lemma.  A  contradiction. 


Thus,  | S (HH)  |  S:  -2L-  -q+1 

max 


Hence,  if  we  define 
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i 


.  e. 


> 


(l>q)  *  1 

£;(p+l»q)  =  (\ (p  >q)+q-l)  •  n 
0  o  max* 


^(p.q) 


.."‘V 

max  ^ 


max 


n  -1 
max 


(q-1) 


for  p,q  *  1  (and  if  F  does  not  satisfy  (1)),  we  will  be  able  to  apply  NSE 

p-1  times  and  obtain  F  ..  G  can  then  be  obtained  as  follows:  If  S(F  ) 

P  P  p-1 

0  SCGj,)  =  <b  for  1  ^  i  p-1,  then  choose  any  variable  y  f  S(F  )  and 

,  .  p-1  _  ^ 

obtain  Gp  from  (Fp-^)  ^  by  normalization;  otherwise,  denoting  U  S(G  ) 

i ""  1  "" 

by  U,  obtain  Fp-1  from  (Fp-1)G  by  normalization.  It  can  be  checked  that 
Gi  ^  1  *  1  *  P  satisfy  the  conditions  of  (2)  of  the  statement  of  the  lemma. 


n 


Consider  a  sequence  of  (nonempty)  sets  Xj,  for  1  £  i  £  p  such  that 

Xi  is  either  a  singleton,  or  is  included  in  U  X  .  Ue  will  call  such 

J<i  J 

a  sequence  of  sets  a  normal  sequence  (of  length  p).  Note  that  the  sequence 
X^,...,Xp  in  Definition  A. 1  is  a  normal  seqdence.  Then 

A.  4  Lemma 

Let  X^...^  be  a  normal  sequence  of  sets  with  the  additional  pro¬ 
perty  that  each  element  of  (j  X  appears  in  at  most  k  elements  of  the 

i=l 

sequence.  Then  if  p  S(k+l)m,  there  exists  a  subset  Y  C  Pi  X  and  an 

i=l  i 

increasing  sequence  of  indices  i(l),i(2) . i(q)  such  that  (1)  q  *  m, 
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(2)  i (1)  -  1,  (3)  X^^  0  Y  is  a  singleton  for  1  S  j  S  q,  (4)  D  Y  =  0 
if  ^  *  *(«[)  and  ^  ^  i(J)  for  1  S  j  S  q,  and  (5)  if  x  <=  Y,  ^  <  J2  <  j3 

andXeXi(J,)’  XeXi(j,)'  tlt"llK-«\(,T 


Proof 

(this  is  a  direct  translation  of  the  proof  of  Lemma  2  of  fHo68]  into 

P 

our  terminology).  Let  U  X^  =  (x^,x2,...].  Without  loss  of  generality 

assume  that  x^  €  X^.  If  m  =  1,  set  Y  =  (x^).  i(l)  =  1,  and  conditions 

are  satisfied.  For  the  inductive  step  two  cases  are  distinguished. 

Case  I.  occurs  in  none  of  the  sets  X  j ,  2  S  j  £  (k+l)™"1  +  1. 

Setting  r  ■  (k+1)  +1,  the  sequence  X2,...,X  is  normal  and  each  element 

r 

occurs  in  at  most  k  of  the  X  ,  2  s:  j  £  r.  If  Z  C  IJ  X.  and  the  sequence 

J  1«2  d 

J (!)»••• »J(q-l)  are  obtained  by  the  inductive  hypothesis,  then  (x^}  U  Z  =  Y 

and  id)  =  1,  1(2)  ■  J  (1) .  ,i(q)  =  j  (q-1)  satisfy  conditions  1-5. 

Case  II.  Assume  that  x^  occurs  in  some  X  j ,  2  S  j  £  (k+l)m""^+l  and 

let  h  be  the  smallest  such  number  j.  Furthermore,  let  V  be  the  set  of 

elements  different  from  x^  and  occurring  in  X2 . X^.  Delete  the 

elements  of  V  from  X^ , . . . ,Xp>  and  delete  those  among  X^X2,...,X 

that  remain  empty.  Let  the  resulting  sequence  be  Y^,...,Y  .  The  length 

of  the  sequence  (X^.X^ . Xp)  is  at  least  p-(k+l)m"1+l. 

There  are  less  than  (k+l)m  ^  distinct  variables  in  X2 . 

each  one  occurring  in  at  most  k-1  of  the  formulas  X, ,X,  . X  . 

1  h  p 


Therefore , 
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r  *  p-(k+l)m"1+l-(k-l)(k+l)m“1,i.e., 
r  2:  (k+l)"*"^! 


m-1 


The  sequence  Y2,...,Yr  is  nomal  and  its  length  is  at  least  (k+1) 

r 

X1  occurs  in  Y2>  Let  Z  E  U  Y.  and  the  sequence  j(l)  =  2,  j(2),.,.t 

j=2  1 

(q-1)  be  obtained  according  to  the  inductive  hypothesis  for  Y, . Y  . 

*  P 

Then  Z  and  i(l)  =  1,  i(2)  =  j  (1) . i(q)  =  j(q-l)  (where  q  *  m),  satisfy 

conditions  1-5.  q 


Let  there  be  given  a  (*,*,k)-formula  F  with  the  proper  nesting 
sequence  G  =  (G^,...,G  )  such  that  G^  is  a  formula  over  $,  As  has 

already  been  remarked  above,  Xx . Xp  (see  Definition  A.l)  is  a  normal 

sequence  of  sets. 

If  p  a  (k+l)m,  then  by  Lemma  A. 4  there  exists  a  set  Y  C  Pi  X  and 

i=l  1 

q  indices  i(j)  for  1  S  j  s  q  such  that  conditions  1-5  hold.  Note  that 

if  m  ■  k't,  then  | Y  J  it  since  no  variable  appears  more  than  k  times 

in  G  (G  is  proper).  In  particular,  consider  only  Z  *  (x^,...,x  )  E  Y 

where  x^ . x^_  are  numbered  in  the  order  of  their  appearance  in  G. 

Note  that  due  to  condition  5  of  Lemma  A. 4,  if  x,  y  6Y  and  y  follows  x 

in  G,  then  x  cannot  appear  again  after  y  in  G.  Let  G  be  as  defined  in 

Definition  A.l.  Then  we  will  let  the  reader  convince  himself  that  G^ 

a 

2 

(hence  also  F^)  is  equivalent  to  K(Z,G')  where  K(Z,z)  is  an  e^-component 
a 

over  $  with  input  variable  z,  and  G*  is  a  certain  formula  over  $a  such 
that  each  variable  of  G*  occurs  at  most  k-1  times. 
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Note  that  in  this  case  we  do  not  know  the  size  of  S(G').  This 
can  be  remedied  in  the  following  way:  There  are  two  cases;  either 
| S (G * ) |  *  1/2* t,  or  not.  In  the  first  case  perform  an  a-merger  on  K 
with  basis  S(G'),  after  which  we  obtain  an  SC  of  an  e“component  K1 
of  length  ^  1/2* t  and  a  formula  G”  (through  the  input  variable)  such 
that  S(G'*)  equals  the  set  of  lateral  variables  of  K' ;  in  the  second 
case  perform  an  a-merger  on  K(Z,G')  with  basis  Z-S(G')  in  which  case  we 
obtain  an  e-component  K'  of  length  ^  1/2* t  with  a  constant  input  operator. 
We  summarize  the  preceding  in  the  following 

A.  5  Lemma 

Let  there  be  given  a  (*,*,k)-formula  F  with  a  proper  nesting  sequence 

2k  •  t 

of  length  p  s  1  composed  of  formulas  over  $.  Then  if  p  s  (k+1)  ,  there 

exists  a  set  Z  C  S(F),  |f|  t,  and  F^  is  either  equivalent  to  an  SC  of 

3  3 

an  et-component  K  over  i  and  a  formula  G  over  ?  such  that  S(G)  is  the 
set  of  lateral  variables  of  K,  and  no  variable  of  G  occurs  more  than 

a 

k-l  times  in  G;  or  to  an  et_component  K  over  $  with  constant  input 
operator. 

Let  there  be  given  a  PC  F  of  the  formulas  F, ,....F  where  r  £  n 
such  that  j  S (F )  |  =  q  and  each  variable  appears  in  at  least  two  among 
Fj,...,F  (i.e.,  a  situation  as  described  in  (1)  of  the  statement 

of  Lemma  A. 3).  We  are  interested  in  obtaining  a  (nonempty)  subset 
X  C  S(F)  such  that  when  the  variables  outside  of  X  have  been  replaced 
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by  the  constant  a,  | S (norm(F. )X) ) |  for  those  F.  where  not  all  variables 

1-  3  1. 

have  been  replaced  with  a  is  equal  or  larger  than  a  predetermined  number 
t  (as  large  as  possible). 

We  could  solve  the  problem  as  follows:  Each  variable  of  S(F)  appears 

in  a  certain  subset  of  the  formulas  F . F  .  The  number  of  such 

r  n 

subsets  is  2  (in  general,  £  2  ax) ;  thus,  we  are  sure  to  find  a  subset  X 

with  |x|  ^  such  that  all  elements  of  X  appear  in  the  same  subset 

_  max 

of  F,  , .  ,F  . 

1  r 

However,  we  can  improve  this  number.  Let  us  construct  the  occurrence 

table  of  F.  The  table  consists  of  rows  corresponding  to  elements  of  S(F), 

and  of  columns  corresponding  to  F,  for  1  s  i  s  r.  The  entry  a  .  is  1 

1  ij 

if  x^  occurs  in  F^  and  0  otherwise.  We  will  try  to  extract  a  subset 

X  C  S(F)  such  that  either  all  variables  of  F^  are  replaced  by  a,  or 

x 

S(norm(F  )  ))  contains  s  elements  (t  will  be  determined  later). 

If  all  columns  in  the  occurrence  table  contain  ^  t  l's,  we  are 
done  and  X  =  S(F).  Suppose  not.  Let  the  column  j  contain  <  t  l's. 

Delete  all  rows  corresponding  to  the  l's  in  column  j  and  column  j 
itself.  Let  the  set  of  variables  corresponding  to  the  remaining  rows 
be  X^.  Consider  the  remainder  of  the  occurrence  table  (i.e.,  minus 
the  deleted  rows  and  column);  and  again  look  for  the  column  with  <  l's. 

If  it  does  not  exist,  we  are  done  and  X  =  X^.  If  such  a  column  exists 
continue.  Now  two  things  can  happen.  Either  at  some  point  we  end  up 
with  a  certain  subset  of  columns,  all  of  whi rh  contain  ^  t  l's,  or  we 
end  up  with  two  columns  that  both  contain  <  t  l's. 


We  shall  see  that  by 
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an  appropriate  choice  of  t,  the  latter  case  cannot  happen.  The  number 
of  1  s  in  the  whole  table  ^  2q  (each  variable  occurs  in  at  least  two 
formulas).  The  smallest  number  of  l's  remaining  after  all  but  two  columns 
have  been  deleted  S  2q-m  where  m  is  largest  possible  number  of  l's  that 


can  be  deleted  in  the  course  of  this  procedure,  m  =  (t-1). (r+r-l+r-2+. 
+3)  =  (t-1)*  ^  ^  (this  corresponds  to  the  case  when  each  deleted 

row  contains  only  l's  and  at  each  stage  t-1  rows  are  deleted).  If,  after 
the  table  is  reduced  to  two  columns,  both  columns  are  to  contain  St  l's 
(both  have  to  contain  the  same  number  of  l's  since  each  variable  occurs 


in  at  least  two  formulas),  then 


or  since  r  s:  n 

max 


t 


t  £ 


where  c  =  (n  +3) (a  -2) 
max  max 


For  large  nmax  this  is  better  than  the  previous  bound.  This  result 
can  be  summarized  in 

A.  6  Lemma 

Let  there  be  given  a  PC  F  of  the  formulas  F^...^  over  $  where 

r  ^  nmax  suc^  t^at  I S (F) |  =  q  and  each  variable  appears  in  at  least 
two  among  F1,...,F  .  Then  if 
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q  s  c  where  til 

V 

we  can  find  a  subset  X  C  S(F)  such  that  Fg  is  equivalent  to  a  PC  of 
the  formulas  . Gg  over  $a  and  S(G^)  S  t  for  1  £  i  £  sir. 

Lemmas  A. 3,  A. 5,  and  A. 6  can  be  combined  into 

A. 7  Lemma 

Let  F  be  an  ($ ,n,k)-formula.  Then  for  any  t  £  1  and  a  £  D  if 

n  «  V(fctl)2k,t,<c-*>;t-g) 

(see  Lemma  A. 6  for  the  value  of  c) ,  there  exists  a  subset  X  C  S(F) 
such  that  either 

X  3 

(1)  F  is  equivalent  to  a  PC  of  the  formulas  F F  over  $a 

3  i  r 

vdiere  r  £  n  ,  each  variable  of  F,  occurs  at  most  k-1  times  in  it. 

max  i  ’ 

and  F^  contains  at  least  t  variables  of  X  or 

X 

(2)  Fg  is  equivalent  to  an  SC  of  an  e^-component  K  over  $a 
with  a  formula  G  over  $a  (through  the  input  variable)  such  that  S(G) 
is  the  set  of  the  lateral  variables  of  K  and  no  variable  occurs  more 
than  k-1  times  in  G;  or  to  an  et“component  K  over  $  with  a  constant 
input  operator. 

A.  8  Lemma 

Let  F  be  a  ($,n,k)-formula.  Then  for  any  t  ^  1  and  a  €  D  if 


n  £  T17  (t  ,k) 
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y 

then  there  exists  a  subset  X  £  S(F)  such  that  is  equivalent  to  an 

SPCeC  over  $  G  such  that  (1)  G  has  ^  nk  components,  (2)  each 

ma  x 

component  is  of  length  ^  t,  and  (3)  the  terminal  components  of  G 
have  constant  input  operators. 

Proof 

^UCtjl)  =  n£„v*  In  this  case  T(F)  has  at  least  one  branch 
'  max 

connected  to  t+1  variable  symbols  (k=l  and  thus  all  variable  symbols 
are  distinct)  at  different  nodes.  This  branch  can  be  converted  into 
an  et-component  with  constant  input  operator.  The  idea  is  illustrated 
in  Fig.  A. 1. 

11_(t,k+i)  =  TL((k+2)2(k+1)* Vc«k)  VfoilQ-c  v 

/  o  »  4  ' 

We  can  apply  Lemma  A. 7.  The  result  is  either  (1)  an  e-component  K 
of  the  correct  length  and  constant  input  operator,  (2)  an  SC  of  an 

g 

e-component  over  $  of  the  correct  length  and  a  formula  to  which  we 
can  apply  the  inductive  hypothesis,  and  (3)  a  PC  of  formulas  to  which 
we  can  apply  the  inductive  hypothesis.  In  each  case  we  obtain  an  SPCeC 
wLth  the  desired  properties.  □ 

A. 9  Lemma 

Let  F  be  a  ($,nk)-formula.  Then  for  any  t  ^  1  and  a  f  D  if 


n  ^  Hg (t ,k) 
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y 

there  exists  a  subset  X  c  S(F)  such  that  F  is  equivalent  to  an  SPCeC 

a 

over  $  G  such  that  (1)  G  has  £  k  components,  (2)  each  component  has  X 
as  the  set  of  its  lateral  variables,  (3)  the  terminal  components  have 
constant  input  operators  and  (4)  |x|  =  t. 

Proof 

Set  Tig  (t  ,k)  =  T|7(s.t,k)  where  s  =^nmaxj  +  ^  nmax  J  +.  ..+  ^nmax^ 

Apply  Lemma  A. 8  to  obtain  a  SPCeC  G'  with  all  components  having  length 
s  S’t.  Since  each  variable  appears  at  most  k  times,  it  can  occur  in 
at  most  k  components,  s  is  the  number  of  nonempty  subsets  of  5  k  elements. 
Thus,  if  the  number  of  variables  is  as  indicated  we  are  sure  to  kind 
in  G'  a  subset  of  t  variables  that  all  occur  in  the  same  set  of  components 
of  G'.  After  performing  an  a-merger  with  this  set  as  basis,  we  obtain 
the  desired  SPCeC  G.  p 


Remarks  on  the  bounds  in  Lemmas  A.3-A.9.  If  T]  is  approximated  by 


P  „ 

n  ,cl>  then  T7  is  inductively  defined  as  follows: 
max  / 


T)7(t,l)  =  n 


max 


*n7(t,k)  =  yn(k+1) 

/  max 


2k-T?(t,k-l) 


■T7(t,k-1) 


for  a  certain  constant  y.  Thus  we  see  that  Tg(t,k)  *  iexp(b,2k)  =  b 
for  k  ^  k(b)  for  any  constant  b  (t  has  not  been  included  in  the  estimate 
because  in  applications  it  is  constant). 


b*  l2k  times 


Ill 


Conversion  of  a  formula  F  where  each  variable  occurs  only  once 
Into  an  equivalent  e-component  by  setting  certain  variables  to  a. 


Fig.  A.  1 
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APPENDIX  B 

THE  LENGTH  OF  THE  MOD  2  SUM  OVER  II  * 

There  is  an  isomorphism  between  the  set  of  formulas  over  II  and  series- 
parellel  contact  networks.  We  assume  the  reader  is  familiar  with  this  model 
as  well  as  with  the  isomorphism  in  question.  In  this  case  if  F  is  a  formula 
over  IT,  then  L(F)  corresponds  to  the  number  of  contacts  in  the  network  corres¬ 
ponding  to  F. 

For  convenience,  we  will  derive  the  result  in  contact  network  terminology. 

Given  a  (series-parallel)  contact  network  C,  a  chain  is  set  of  contacts 
such  that  when  they  are  all  closed,  C  conducts  (we  will  say  "C  is  1");  a  cut 
set  is  a  set  of  contacts  such  that  when  they  are  all  open,  C  does  not  conduct 
(we  will  say  "C  is  0").  In  the  obvious  way,  we  define  minimal  chain,  minimal 
cut  set  (i.e.,  when  one  contact  is  deleted  the  corresponding  property  does  not 
hold). 


B.  1  Lenina 

Given  a  contact  network  C  and  any  minimal  chain  and  minimal  cut  set,  their 
intersection  is  a  singleton. 


This  result  is  due  to  Khrapchenko  [Kh71]. 
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Proof 

By  induction  on  the  number  m  of  contacts  in  C.  For  m  =  1  the  assertion 
is  obviously  true.  If  m  >  1,  C  must  be  either  a  series  combination  of  smaller 
networks  and  or  a  parallel  combination  of  smaller  networks  and  C^. 

In  each  case  it  is  simple  to  establish  the  lemma.  □ 


Suppose  now  we  have  a  contact  network  S  that  represents  0  x..  Let  m. 

i=l  ^ 

denote  the  number  of  contacts  labeled  with  x.  or  x.  .  Then  we  are  interested 
n  J  J 

in  £  m.. 


J-l 


Consider  n-tuples  (a^  for  1  ^  i  ^  n  and  a£  €  (0,1).  An  n-tuple  of  this 
kind  will  be  called  even  if  it  has  an  even  number  of  l's, otherwise  it  is  odd. 
Obviously  S  must  be  1  on  odd  n-tuples  and  0  on  even  ones. 

Consider  an  arbitrary  odd  n-tuple  a  ®  (a^...^ . a^)  and  an  even 

n-tuple  b  =  (b^,. • .  ,b^,. . . jb^)  at  Haraning  distance  1  from  a.  If  b^  =  a^,  then 

all  other  components  of  a  and  b,  are  equal.  will  denote  the  n-tuple  with 

a  single  1  in  the  i  ^  place.  Then  we  will  write  b  =  a  0  e. . 

“  - 

To  each  odd  n-tuple  a  we  can  assign  a  minimal  chain  c(a)  (consisting  of 
a  subset  of  contacts  of  S  that  are  closed  at  a  and  that  do  form  a  minimal 
chain);  similarly,  to  each  even  n-tuple  b  we  can  assign  a  minimal  cut  set 
s(b)  (consisting  of  a  set  of  contacts  of  S  that  are  open  at  b  and  that  do  form 
a  minimal  cut  set). 

Let  a  be  odd,  b  =  a  0  e^  even.  Then  by  Lemma  B.l,  c(a)  H  s(b)  is  a 
singleton;  in  fact,  it  is  easy  to  verify  that  it  must  be  a  contact  labcle'1 
either  with  x^  or  x^ . 
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We  build  now  Tables  I  and  II.  The  rows  of  Table  I  correspond  to  odd 


n-tuples  while  those  of  Table  II  correspond  to  even  n- tuples.  Thus  both 
n*“  1 

have  2  rows.  The  columns  of  both  tables  correspond  to  the  variable 

for  1  £  i  £  n.  The  entry  a(a,j)  in  Table  I  is  c(a)  D  s(a  &  £.).  This  entry 
will  be  represented  by  a  number  between  1  and  Mj. 

Let  t^j  denote  the  number  of  times  contact  number  i  (among  those  labeled 
with  Xj  or  Xj)  appears  in  column  j  of  Table  I.  Then 


m 


'j 

*  ‘i,  "  2 

i=l  1J 


n-1 


(B.l) 


The  entry  P(b,j)  of  Table  II  is  s(b)  fi  c(b  $  e^).  (B.l)  again  holds. 

Construct  now  Table  III.  The  rows  of  Table  III  correspond  to  all  possible 
pairs  (a,b)  where  a  and  b  are  odd  and  even  n-tuples  respectively.  The  columns 
of  Table  III  again  correspond  to  variables.  An  entry  of  Table  III  is  Y(a>Jl>j) 

<<*(*. j), 

Consider  now  the  diagonal  entries  of  Table  III  (i.e.  ,  (a,|J)  such  that 
a  =  p).  Let  (Ot(a,j),  P(u,j))  be  such  an  entry.  Then  a(a,j)  =  P(b,j)  = 
c(a)  0  s(b).  Thus,  by  Lemma  B.l,  there  can  be  only  one  such  entry  in  a  row. 

n  mj  2 

The  number  of  diagonal  entries  is  S  E  t  .  Thus, 

j=l  i-1  J 


n 

E 


m . 


i  tj, 

J-l  i=l  J 


(B.  2 ) 


Combining  a  version  of  Cauchy's  inequality 


m. 


m 


—  (  ^  t  )  ^  E 

j  i=l  ij 


i=l 


'ij 
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with  (B.l)  and  (B.2),  we  obtain 


Z  —  ^  1 
J-l  mJ 


This  time  we  apply  the  inequality 

n  n 


(  Z  in,).  (  Z  -L  )  2  „2 
j-l  J  j-l  ”j 

(for  both  inequalities  see,  e.g. ,  [Mt64]p.  9),  and  obtain  the  desired  result 


E  m.  ^  n2 

j-l  J 
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